-- Logs begin at Thu 2022-05-26 16:48:44 UTC, end at Thu 2022-05-26 18:28:42 UTC. -- May 26 17:38:13.191980 dv-cinder-836073-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 26 17:38:13.200726 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 26 17:38:13.202731 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 26 17:38:13 2022] *** May 26 17:38:13.202731 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 26 17:38:13.202731 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 26 17:38:13.202731 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: nodename: dv-cinder-836073-5-iscsi May 26 17:38:13.202731 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: machine: x86_64 May 26 17:38:13.202731 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: clock source: unix May 26 17:38:13.202731 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: pcre jit disabled May 26 17:38:13.202731 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: detected number of CPU cores: 4 May 26 17:38:13.202731 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: current working directory: / May 26 17:38:13.202731 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: detected binary path: /usr/bin/uwsgi-core May 26 17:38:13.202731 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: your processes number limit is 47680 May 26 17:38:13.203491 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: your memory page size is 4096 bytes May 26 17:38:13.203491 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: detected max file descriptor number: 2048 May 26 17:38:13.203491 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: lock engine: pthread robust mutexes May 26 17:38:13.203491 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: thunder lock: enabled May 26 17:38:13.203491 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 26 17:38:13.203491 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 26 17:38:13.248466 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: Python main interpreter initialized at 0x56296ba289c0 May 26 17:38:13.248466 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: python threads support enabled May 26 17:38:13.248466 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: your server socket listen backlog is limited to 100 connections May 26 17:38:13.248466 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: your mercy for graceful operations on workers is 90 seconds May 26 17:38:13.248844 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: mapped 403077 bytes (393 KB) for 2 cores May 26 17:38:13.248922 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: *** Operational MODE: preforking *** May 26 17:38:13.249170 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: *** uWSGI is running in multiple interpreter mode *** May 26 17:38:13.249170 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: spawned uWSGI master process (pid: 92071) May 26 17:38:13.249383 dv-cinder-836073-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 26 17:38:13.250426 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: spawned uWSGI worker 1 (pid: 92082, cores: 1) May 26 17:38:13.250426 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: spawned uWSGI worker 2 (pid: 92083, cores: 1) May 26 17:38:13.250426 dv-cinder-836073-5-iscsi devstack@keystone.service[92071]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 26 17:38:19.007819 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 17:38:19.007819 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: from cryptography.utils import int_from_bytes May 26 17:38:19.009207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 17:38:19.009207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: from cryptography.utils import int_from_bytes May 26 17:38:19.010544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 17:38:19.010544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: from cryptography.utils import int_from_bytes May 26 17:38:19.011045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 17:38:19.011045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: from cryptography.utils import int_from_bytes May 26 17:38:19.058416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 17:38:19.058588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 17:38:19.058776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] command line args: [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 17:38:19.059070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 17:38:19.059248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ================================================================================ {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 17:38:19.059500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] admin_token = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.059745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] config_dir = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.059934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.060017 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 17:38:19.060100 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] config_source = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.060177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 17:38:19.060281 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] debug = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.060367 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] command line args: [] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 17:38:19.060479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.060793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 17:38:19.060793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ================================================================================ {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 17:38:19.060984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.061083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] admin_token = **** {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.061190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.061285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] config_dir = [] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.061389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.061485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.061585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.061686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] config_source = [] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.061797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.061892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] debug = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.061996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_config_append = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.062092 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.062224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.062321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.062419 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_dir = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.062521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.062627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_file = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.062736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.062841 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.062960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.063034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.063165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] list_limit = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.063242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.063362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] log_config_append = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.063436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.063559 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.063638 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.063777 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] log_dir = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.063853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.063968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] log_file = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.064041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.064155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.064240 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.064318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.064441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.064521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.064597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.064711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.064837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.064917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.064993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.065108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.065189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.065260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.065410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.065493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.065647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.065730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.065886 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.066123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.066207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.121/identity {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.066351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] publish_errors = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.066432 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.066582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.066659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.066810 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.066910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.067025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.067136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.067265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.067459 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.067601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.121/identity {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.067715 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] standard_threads = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.067826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] publish_errors = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.067938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.068134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.068255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.068376 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.068497 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.068615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.068726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] use_journal = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.068836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.068946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] use_json = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.068946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] use_stderr = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.069160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.069279 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] use_syslog = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.069392 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] standard_threads = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.069502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.069619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.069737 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.069857 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.069969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.070079 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.070079 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] use_journal = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.070298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.070972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] use_json = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.071227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.071345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] use_stderr = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.071538 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.071725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] use_syslog = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.071878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.071992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:38:19.071992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.072213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.072331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.072443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.072562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.072680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.072810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.072930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.073049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.073167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.073278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.073388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.073507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.073628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.073746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.073858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.073975 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.074087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.074277 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.074397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.074510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.074632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.074745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.074883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.074999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.075110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.075221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.075331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.075441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.075559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.075679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.075798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.075917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.076029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.076146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.076266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.076379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.076490 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.076601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.076719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.076831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.076950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.077063 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.077174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.077291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.077402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.077512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.077634 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.077755 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.077868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.077978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.078088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.078204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.078324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.078427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.078521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.078612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.078710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.078806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.078948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.079045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.079148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.079249 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.079350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.079445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.079566 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.079711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.079836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.079951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.080069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.080172 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.080291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.080404 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.080515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.080626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.080736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.080853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.080975 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.081088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.081205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.081318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.081429 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.081549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.081673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.081787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.081899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.082013 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.082130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.082243 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.082355 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.082474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.082588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.082708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.082833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.082950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.083079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.083206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.083326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.083429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.083540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.083651 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.083762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.083885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.083998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.084110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.084221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.084338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.084450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.084562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.084674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.084786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.084897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.085008 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.085129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.085243 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.085366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.085488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.085600 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.085600 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.085821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.086007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.086123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.086357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.086476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.086476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.086476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.086803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.086803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.087069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.087186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.087299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.087410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.087534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.087650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.087762 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.087873 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.087992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.088105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.088225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.088339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.088451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.088563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.088681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.088794 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.088918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.089049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.089164 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.089277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.089395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.089507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.089619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.089738 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.089859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.089978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.090097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.090217 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.090331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.090451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.090565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.090685 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.090799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.090932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.091029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.091131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.091232 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.091328 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.091424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.091518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.091613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.091714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.091810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.091904 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.091998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.092091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.092186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.092287 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.092383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.092478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.092571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.092770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.092770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.093006 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.093109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.093206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.093299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.093393 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.093499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.093603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.093699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.093810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.093906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.094000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.094000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.094180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.094277 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.094371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.094471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.094589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.external = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.094701 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.094834 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.094951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.095062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.095177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.095289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.095401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.password = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.095528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.095644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.token = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.095784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.095907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.096022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] auth.external = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.096134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.096246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.096358 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.096478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.096595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.096708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.096821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.096942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] auth.password = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.097060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.097184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] auth.token = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.097307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.097422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.097535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.097647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.097761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.097890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.098014 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.098129 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.098241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.098354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.098467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.098651 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.098777 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.098941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.099055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.099168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] credential.caching = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.099294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.099410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.099523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.099636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.099749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.099862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.099983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.100097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.100215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.100330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.100330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.100541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.100663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.100784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.100900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.101012 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.101125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.101237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.101237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.101461 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.101574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.101678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.101898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.102026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.102133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.102231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.102325 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.102478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.102601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.102714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.102857 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.102979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.103077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] federation.caching = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.103212 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.103317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.103469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.103583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.103705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.103849 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.103952 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.104046 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.104175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.104287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.104413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.104515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.104656 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.104758 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.104891 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.105009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.105112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.105209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.105357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.105476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.105590 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.105712 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.105827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.105940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.caching = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.106053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.106166 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.106278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.106400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.106515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.106628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.106741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.106887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.107131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.107257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.107373 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.107486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.107598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.107711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.107838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.108496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.108617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.108731 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.108844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.108844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.109069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.109184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.109297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.109297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.109583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.109706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.109822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.109935 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.110048 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.110162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.110285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.110400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.110514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.110634 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.110749 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.110878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.111007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.111125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.111245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.111360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.111474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.111587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.111708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.111937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.112050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.112164 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.112277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.112400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.112525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.112640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.112753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.112866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.112980 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.113102 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.113217 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.113331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.113445 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.113565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.113680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.113801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.113917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.114030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.114143 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.114257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.114371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.114494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.114609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.114722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.114850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.114968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.115081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.115208 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.115324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.115437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.115437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.115665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.115780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.115905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.116021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.116140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.116254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.116368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.116481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.116608 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.116725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.116838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.116952 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.117069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.117069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.117978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.118080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.118194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.118293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.118388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.118482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.118603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.118720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.118896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.118896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.119115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.119115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.119340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.119457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.119573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.119689 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.119802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.119916 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.120038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.120153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.120267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.120380 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.120493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.120607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.120730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.120846 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.120960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.121073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.121187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.121299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.121421 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.121536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.121650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.121763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.121878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.121991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.122180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.122298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.122412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.122526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.122675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.122793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.122938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.123055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.123169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.123313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.123430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.123544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.123666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.123781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.123896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.124010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.124122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.124243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.124359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.124507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.124623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.124736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.124850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.124974 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.125089 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.125203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.125316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.125429 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.125542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.125664 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.125786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.125901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.126016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.126129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.126242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.126242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.126464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.126578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.126690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.126690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.126957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.127089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.127206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.127320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.127433 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.127547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.127661 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.127783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.127898 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] resource.caching = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.128011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.128127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.128240 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.128353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.128475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] role.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.128590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.128704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] role.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.128817 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.128931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.129044 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.129168 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.129284 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.129398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.129511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.129624 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.129738 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.129860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.129976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.130089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.130204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] role.caching = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.130317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.130430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] role.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.130552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.130668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.130781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.130910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.131025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.131139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.131266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.131383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.131497 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.131611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.131724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.131838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.131962 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.132078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.132078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.132291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.132411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.132517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.132637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.132752 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.133460 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.133711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.133830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.133943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.134060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.134174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.134288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.134403 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.134516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.134639 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.134755 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.134911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.134911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.135128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.135241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.135368 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.135486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.135601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.135714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.135828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.135941 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.136069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.136186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.136299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.136413 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.136527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.136640 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.136763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.136879 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.136992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.137110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.137225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.137338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.137461 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.137577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.137690 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.137803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.137803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.138022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.138145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.138261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.138375 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.138488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.138603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.138716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.138716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.138956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.139071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] token.caching = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.139185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.139299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.139412 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.139538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.139655 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.139769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.139882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.139882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.140094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.140216 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.140332 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.140445 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.140559 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.140673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.140787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.140909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.141025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.141025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.141244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.141359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.141472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.connection = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.141599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.141728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.141844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.141958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.142071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.142185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.142298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.142420 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.142535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.142648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.142763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.142917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.142917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.143207 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.143326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.143441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.connection = **** {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.143564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.143688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.143805 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.143920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.144046 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.144162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.144276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.144389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.144494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.144607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.144729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.144845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.144959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.145073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.145186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.145299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.145422 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.145537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.145650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.145764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.145878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 17:38:19.145993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.145993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.145993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.146314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.146434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.146548 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.146786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.147046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:38:19.147219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92082) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 17:38:19.177451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.177678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.177904 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.177988 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.178191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.178424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.187340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.187972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.216976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.217206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.217574 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.217812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.271452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.271739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.324905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.325136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.326090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.326331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.393459 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.393681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.393888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.393980 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.394109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.394194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.394326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.394420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.394541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.394625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.394806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.395048 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.419800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.420038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.420271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.420504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.420952 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.421221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.421303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.421520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.421601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.421745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.421966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.422183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.422397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.422612 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.434542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.435108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.448099 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.448324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.448547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.448769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.448985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.449200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.449415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.449878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.450119 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.450352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.450583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.450806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.451050 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.451290 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92082) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:38:19.469243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.469444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.469592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 17:38:19.469736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 17:38:19.469876 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.469985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.470125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 17:38:19.470289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 17:38:19.470429 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.470525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.470653 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.470732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.470884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.471040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.471125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.471250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.483074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.483360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.483476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.483630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.483768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.483912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.484042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.484326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.492061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.493839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.496193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.496476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.496757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.497042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.497321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.497402 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.497579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.497664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.497845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.497930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.498147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.498229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.498430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.498521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.498745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.498853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.499055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.499138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.499435 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.499715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.500034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.500311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.526220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.527700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.527836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.528250 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.529491 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.529775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.533708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.533990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.534276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.535165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.535451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.535734 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.542252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.542536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.544089 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.544388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.547583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.547874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.548157 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.548571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.548842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.548953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.549117 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.549384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.549470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.549652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.549733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.549939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.550214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.550490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.550761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.551058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.569261 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.570768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.571074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.574222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.575801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.577854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.578147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.579475 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.579771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.583366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.585026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.587125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.588961 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.590772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.591099 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.591492 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.591785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.592071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.592349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.592773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.593085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.593370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.593659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.593943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.594221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.612027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.612309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.612577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.612840 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.614023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.614308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.614575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.614857 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.621654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.623394 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.623668 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.623935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.625673 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.625954 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.626718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.627023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.627303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.627581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.627859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.629056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.629341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.629624 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.629908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.630187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.641415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.641689 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.641967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.642251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.642530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.642802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.643114 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.644026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.644308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.644603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.644885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.645169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.645446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.645741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.661139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.665223 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.665399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.668988 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.669130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.670736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.672354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.672457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.672668 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.672949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.673973 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.675652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.675944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.676234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.681038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.682647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.684378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.685901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.686008 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.686193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.686473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.686745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.689382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.689674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.689954 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.690228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.696274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.696589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.696870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.699737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.700027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.700409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.704994 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.705272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.705549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.705821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.706089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.706357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.706636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.706925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.707204 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.707477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.707745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.708009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.708277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.708355 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92082) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:38:19.708542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.708631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.708903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.709181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.709453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.709722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.710009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.710274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.710549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.710845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.711331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.711613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.711882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.712200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92082) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:38:19.749616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56296ba289c0 pid: 92083 (default app) May 26 17:38:19.752340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56296ba289c0 pid: 92082 (default app) May 26 17:38:19.752874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a6a61-a4a7-4bab-b117-ed4f32e81d63 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:19.753164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a6a61-a4a7-4bab-b117-ed4f32e81d63 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:19.753435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a6a61-a4a7-4bab-b117-ed4f32e81d63 None None] PATH_INFO: `/v3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:19.757813 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1/1] 10.222.0.121 () {50 vars in 796 bytes} [Thu May 26 17:38:19 2022] GET /identity/v3/ => generated 256 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:29.125353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a42fd-80c8-4b22-85e1-1ef95a886884 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:29.125900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a42fd-80c8-4b22-85e1-1ef95a886884 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:29.125900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a42fd-80c8-4b22-85e1-1ef95a886884 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:29.130355 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1/2] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:29 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:29.136213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1dc51-7995-4add-994b-e387bbfa7b32 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:29.136660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1dc51-7995-4add-994b-e387bbfa7b32 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:29.136753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1dc51-7995-4add-994b-e387bbfa7b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:29.204558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: INFO dbcounter [None req-63b1dc51-7995-4add-994b-e387bbfa7b32 None None] Registered counter for database keystone May 26 17:38:29.226736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG oslo_db.sqlalchemy.engines [None req-63b1dc51-7995-4add-994b-e387bbfa7b32 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92083) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 17:38:29.233139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writer thread running {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 17:38:29.422505 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG passlib.handlers.bcrypt [None req-63b1dc51-7995-4add-994b-e387bbfa7b32 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92083) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 17:38:29.422946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG passlib.handlers.bcrypt [None req-63b1dc51-7995-4add-994b-e387bbfa7b32 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92083) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 17:38:29.480963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-63b1dc51-7995-4add-994b-e387bbfa7b32 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:29.482556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-63b1dc51-7995-4add-994b-e387bbfa7b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:29.612154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 2/3] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:29 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 26 17:38:29.776353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a42fd-80c8-4b22-85e1-1ef95a886884 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:29.845895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: INFO dbcounter [None req-5d2a42fd-80c8-4b22-85e1-1ef95a886884 None None] Registered counter for database keystone May 26 17:38:29.866290 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG oslo_db.sqlalchemy.engines [None req-5d2a42fd-80c8-4b22-85e1-1ef95a886884 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92082) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 17:38:29.873222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writer thread running {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 17:38:29.995592 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a42fd-80c8-4b22-85e1-1ef95a886884 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:30.000198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961500e-4974-45b0-8327-af9a912f7b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:30.000862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961500e-4974-45b0-8327-af9a912f7b9e admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:30.001142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961500e-4974-45b0-8327-af9a912f7b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:30.001415 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961500e-4974-45b0-8327-af9a912f7b9e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:30.009632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961500e-4974-45b0-8327-af9a912f7b9e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:30.062395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f961500e-4974-45b0-8327-af9a912f7b9e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 17:38:30.062655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961500e-4974-45b0-8327-af9a912f7b9e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:30.072634 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 2/4] 10.222.0.121 () {58 vars in 1329 bytes} [Thu May 26 17:38:29 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 17:38:30.078980 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1dc51-7995-4add-994b-e387bbfa7b32 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:30.093596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1dc51-7995-4add-994b-e387bbfa7b32 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:30.097982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee14e24-f1df-440a-a9e7-72e1b5b8cdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:30.099139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee14e24-f1df-440a-a9e7-72e1b5b8cdbe admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:30.099424 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee14e24-f1df-440a-a9e7-72e1b5b8cdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:30.099699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee14e24-f1df-440a-a9e7-72e1b5b8cdbe admin admin] PATH_INFO: `/v3/projects/a01a22dec10b499785fdfbdf0a6fcb47` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:30.101743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee14e24-f1df-440a-a9e7-72e1b5b8cdbe admin admin] RBAC: Authorizing `identity:get_project(project_id=a01a22dec10b499785fdfbdf0a6fcb47)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:30.268089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0ee14e24-f1df-440a-a9e7-72e1b5b8cdbe admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 17:38:30.268717 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee14e24-f1df-440a-a9e7-72e1b5b8cdbe admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:30.273307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 3/5] 10.222.0.121 () {56 vars in 1212 bytes} [Thu May 26 17:38:30 2022] GET /identity/v3/projects/a01a22dec10b499785fdfbdf0a6fcb47 => generated 345 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:32.487024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711833b-16a9-4c3c-8ccf-3cba8fe945ba None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:32.487593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711833b-16a9-4c3c-8ccf-3cba8fe945ba None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:32.487593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711833b-16a9-4c3c-8ccf-3cba8fe945ba None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:32.492357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 3/6] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:32 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:32.498106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9079e-9f90-4986-8b15-a5a38000030b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:32.498501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9079e-9f90-4986-8b15-a5a38000030b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:32.498661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9079e-9f90-4986-8b15-a5a38000030b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:32.518761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-53a9079e-9f90-4986-8b15-a5a38000030b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:32.520323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-53a9079e-9f90-4986-8b15-a5a38000030b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:32.528313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 4/7] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:32 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 26 17:38:32.551095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711833b-16a9-4c3c-8ccf-3cba8fe945ba None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:32.564041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711833b-16a9-4c3c-8ccf-3cba8fe945ba None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:32.568292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51446a86-d0ab-4266-9ddf-069d87616827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:32.568930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51446a86-d0ab-4266-9ddf-069d87616827 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:32.569214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51446a86-d0ab-4266-9ddf-069d87616827 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:32.569488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51446a86-d0ab-4266-9ddf-069d87616827 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:32.578292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51446a86-d0ab-4266-9ddf-069d87616827 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:32.580108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51446a86-d0ab-4266-9ddf-069d87616827 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:32.589733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 4/8] 10.222.0.121 () {58 vars in 1329 bytes} [Thu May 26 17:38:32 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 17:38:32.595521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9079e-9f90-4986-8b15-a5a38000030b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:32.604622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9079e-9f90-4986-8b15-a5a38000030b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:32.609238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5b195-295b-4342-8f4f-40a0a9e6174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:32.610112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5b195-295b-4342-8f4f-40a0a9e6174c admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:32.610393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5b195-295b-4342-8f4f-40a0a9e6174c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:32.610666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5b195-295b-4342-8f4f-40a0a9e6174c admin admin] PATH_INFO: `/v3/users/9845c1dd09f34a0aa117c5e212e20304` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:32.612093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5b195-295b-4342-8f4f-40a0a9e6174c admin admin] RBAC: Authorizing `identity:get_user(user_id=9845c1dd09f34a0aa117c5e212e20304)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:32.613944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5b195-295b-4342-8f4f-40a0a9e6174c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:32.615592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 5/9] 10.222.0.121 () {56 vars in 1203 bytes} [Thu May 26 17:38:32 2022] GET /identity/v3/users/9845c1dd09f34a0aa117c5e212e20304 => generated 248 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:34.937665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f253223-d198-4a10-a762-2a6c9bd90261 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:34.938217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f253223-d198-4a10-a762-2a6c9bd90261 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:34.938217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f253223-d198-4a10-a762-2a6c9bd90261 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:34.939585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 5/10] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:34.945511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02d988-3d10-411c-adf3-43d3d1d3c418 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:34.946037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02d988-3d10-411c-adf3-43d3d1d3c418 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:34.946037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02d988-3d10-411c-adf3-43d3d1d3c418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:34.962660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1b02d988-3d10-411c-adf3-43d3d1d3c418 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:34.963854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1b02d988-3d10-411c-adf3-43d3d1d3c418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:35.044439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 6/11] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:35.067765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f253223-d198-4a10-a762-2a6c9bd90261 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:35.097385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5663b0-cb83-403c-a9a9-c885e0600a63 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:35.097683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5663b0-cb83-403c-a9a9-c885e0600a63 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:35.097967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5663b0-cb83-403c-a9a9-c885e0600a63 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:35.099357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 7/12] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:38:35.108060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b30202-8a72-4e0a-a99d-37ce316a84bf None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:35.108229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b30202-8a72-4e0a-a99d-37ce316a84bf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:35.108510 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b30202-8a72-4e0a-a99d-37ce316a84bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:35.125992 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-53b30202-8a72-4e0a-a99d-37ce316a84bf None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:35.127244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-53b30202-8a72-4e0a-a99d-37ce316a84bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:35.144858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:35.199882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 8/13] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:38:35.216655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f253223-d198-4a10-a762-2a6c9bd90261 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:35.221989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b30202-8a72-4e0a-a99d-37ce316a84bf None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:35.251898 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668926d-3934-44e5-a5b5-d46913fd58b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:35.252704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668926d-3934-44e5-a5b5-d46913fd58b0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:35.252986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668926d-3934-44e5-a5b5-d46913fd58b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:35.253259 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668926d-3934-44e5-a5b5-d46913fd58b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:35.265302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668926d-3934-44e5-a5b5-d46913fd58b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-8668926d-3934-44e5-a5b5-d46913fd58b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:35.277501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:35.278445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:38:35.279617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 6/14] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:38:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:38:35.284496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668926d-3934-44e5-a5b5-d46913fd58b0 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:35.327802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b30202-8a72-4e0a-a99d-37ce316a84bf None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:35.365517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09446d6a-f6ce-4a1b-910b-5d5d90f69832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:35.366176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09446d6a-f6ce-4a1b-910b-5d5d90f69832 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:35.366455 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09446d6a-f6ce-4a1b-910b-5d5d90f69832 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:35.367524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09446d6a-f6ce-4a1b-910b-5d5d90f69832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:35.390738 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:35.429662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:35.440897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09446d6a-f6ce-4a1b-910b-5d5d90f69832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:35.442417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09446d6a-f6ce-4a1b-910b-5d5d90f69832 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:35.502994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668926d-3934-44e5-a5b5-d46913fd58b0 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:35.536328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f659b76-b03a-48e3-b405-59ea813eb911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:35.536998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f659b76-b03a-48e3-b405-59ea813eb911 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:35.537287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f659b76-b03a-48e3-b405-59ea813eb911 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:35.537561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f659b76-b03a-48e3-b405-59ea813eb911 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:35.538310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f659b76-b03a-48e3-b405-59ea813eb911 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:35.539527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f659b76-b03a-48e3-b405-59ea813eb911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:35.540796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f659b76-b03a-48e3-b405-59ea813eb911 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:35.542293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 9/15] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:38:35 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:38:35.547902 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09446d6a-f6ce-4a1b-910b-5d5d90f69832 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:35.553389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 7/16] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:38:35 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:38:35.559057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f659b76-b03a-48e3-b405-59ea813eb911 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:35.600399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:35.648904 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09446d6a-f6ce-4a1b-910b-5d5d90f69832 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:35.656493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:35.662615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f659b76-b03a-48e3-b405-59ea813eb911 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:35.684005 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeb78c-2299-48a4-a22f-8d79147eb82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:35.684803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeb78c-2299-48a4-a22f-8d79147eb82f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:35.685085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeb78c-2299-48a4-a22f-8d79147eb82f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:35.685361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeb78c-2299-48a4-a22f-8d79147eb82f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:35.696383 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeb78c-2299-48a4-a22f-8d79147eb82f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:35.697745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e4e72-9f69-46c6-955b-ea51d03b9dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:35.697862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeb78c-2299-48a4-a22f-8d79147eb82f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:35.698690 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e4e72-9f69-46c6-955b-ea51d03b9dfe None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:35.699004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e4e72-9f69-46c6-955b-ea51d03b9dfe None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:35.699280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e4e72-9f69-46c6-955b-ea51d03b9dfe None admin] PATH_INFO: `/v3/users/9845c1dd09f34a0aa117c5e212e20304` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:35.700917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e4e72-9f69-46c6-955b-ea51d03b9dfe None admin] RBAC: Authorizing `identity:get_user(user_id=9845c1dd09f34a0aa117c5e212e20304)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:35.702349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e4e72-9f69-46c6-955b-ea51d03b9dfe None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:35.703998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 8/17] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:35 2022] GET /identity/v3/users/9845c1dd09f34a0aa117c5e212e20304 => generated 248 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:38:35.709267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e4e72-9f69-46c6-955b-ea51d03b9dfe None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-4eaeb78c-2299-48a4-a22f-8d79147eb82f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:38:35.719360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:35.720218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:35.721180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:38:35.721180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:38:35.721180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:38:35.721180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:38:35.721180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 10/18] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:38:35.727234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeb78c-2299-48a4-a22f-8d79147eb82f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:35.811574 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e4e72-9f69-46c6-955b-ea51d03b9dfe None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:35.833388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeb78c-2299-48a4-a22f-8d79147eb82f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:35.836225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:35.846181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada445a8-cc95-4474-a575-942aedc76884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:35.848546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada445a8-cc95-4474-a575-942aedc76884 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:35.848546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada445a8-cc95-4474-a575-942aedc76884 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:35.848546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada445a8-cc95-4474-a575-942aedc76884 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:35.848785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada445a8-cc95-4474-a575-942aedc76884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:35.850269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada445a8-cc95-4474-a575-942aedc76884 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:35.853211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 9/19] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:38:35 2022] GET /identity/v3/domains/default => generated 200 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:38:35.858921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada445a8-cc95-4474-a575-942aedc76884 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:35.873327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184261b-1f8d-4bd8-b543-fc2fb8251d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:35.874003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184261b-1f8d-4bd8-b543-fc2fb8251d73 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:35.874287 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184261b-1f8d-4bd8-b543-fc2fb8251d73 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:35.874559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184261b-1f8d-4bd8-b543-fc2fb8251d73 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:35.875329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184261b-1f8d-4bd8-b543-fc2fb8251d73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:35.875631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184261b-1f8d-4bd8-b543-fc2fb8251d73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:35.877038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184261b-1f8d-4bd8-b543-fc2fb8251d73 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:35.892437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:35.896575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 11/20] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:38:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:38:35.971134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada445a8-cc95-4474-a575-942aedc76884 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:36.011889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4cebb0-9554-47a0-b70a-da0e5ca15037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:36.012562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cebb0-9554-47a0-b70a-da0e5ca15037 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:36.012840 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cebb0-9554-47a0-b70a-da0e5ca15037 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:36.013109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cebb0-9554-47a0-b70a-da0e5ca15037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:36.013958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4cebb0-9554-47a0-b70a-da0e5ca15037 None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.domain.id=default, user.id=9845c1dd09f34a0aa117c5e212e20304) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:36.014264 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4cebb0-9554-47a0-b70a-da0e5ca15037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:36.016313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4cebb0-9554-47a0-b70a-da0e5ca15037 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:36.038864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 10/21] 10.222.0.121 () {56 vars in 1327 bytes} [Thu May 26 17:38:35 2022] GET /identity/v3/role_assignments?user.id=9845c1dd09f34a0aa117c5e212e20304&scope.domain.id=default&role.id=4f574961798246a4b12ad2276e67685d => generated 234 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:38:38.489348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f43a5-b752-4e67-afba-635bdf80b399 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:38.490356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f43a5-b752-4e67-afba-635bdf80b399 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:38.490866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f43a5-b752-4e67-afba-635bdf80b399 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:38.493432 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 12/22] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:38 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:38.499270 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db349d2-3a50-4bb0-82d0-db8fea152435 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:38.499711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db349d2-3a50-4bb0-82d0-db8fea152435 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:38.499809 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db349d2-3a50-4bb0-82d0-db8fea152435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:38.521346 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG passlib.handlers.bcrypt [None req-9db349d2-3a50-4bb0-82d0-db8fea152435 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92082) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 17:38:38.522084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG passlib.handlers.bcrypt [None req-9db349d2-3a50-4bb0-82d0-db8fea152435 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92082) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 17:38:38.580378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-9db349d2-3a50-4bb0-82d0-db8fea152435 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:38.582347 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9db349d2-3a50-4bb0-82d0-db8fea152435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:38.703733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 11/23] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:38.756766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f43a5-b752-4e67-afba-635bdf80b399 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:38.894170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb8be1-6e0d-4ba4-a357-53ca2c609770 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:38.894853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb8be1-6e0d-4ba4-a357-53ca2c609770 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:38.895361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb8be1-6e0d-4ba4-a357-53ca2c609770 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:38.897218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 12/24] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:38 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:38.903862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414ee57-28b2-451f-869a-be9fb4eb2e88 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:38.904672 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414ee57-28b2-451f-869a-be9fb4eb2e88 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:38.905262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414ee57-28b2-451f-869a-be9fb4eb2e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:38.923577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f43a5-b752-4e67-afba-635bdf80b399 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:38.945678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c414ee57-28b2-451f-869a-be9fb4eb2e88 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:38.947951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c414ee57-28b2-451f-869a-be9fb4eb2e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:38.994921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:38.997857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbed2d-9a06-4b6a-952e-e3bd61a6d7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:38.998691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbed2d-9a06-4b6a-952e-e3bd61a6d7ab None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:38.999004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbed2d-9a06-4b6a-952e-e3bd61a6d7ab None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:38.999274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbed2d-9a06-4b6a-952e-e3bd61a6d7ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:39.020051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbed2d-9a06-4b6a-952e-e3bd61a6d7ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:39.022037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbed2d-9a06-4b6a-952e-e3bd61a6d7ab None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-95cbed2d-9a06-4b6a-952e-e3bd61a6d7ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:38:39.058568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:39.059485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:39.060744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:38:39.060744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:38:39.060744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:38:39.060744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:38:39.060744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 13/25] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:38:39.065670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbed2d-9a06-4b6a-952e-e3bd61a6d7ab None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:39.111209 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:39.133621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 13/26] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:38:39.136791 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2008a-a509-4af3-8928-a42896a52fd7 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:39.137083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2008a-a509-4af3-8928-a42896a52fd7 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:39.137360 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2008a-a509-4af3-8928-a42896a52fd7 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:39.138892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 14/27] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:39 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:39.151410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19227126-720b-4548-ba05-fdbc4b572002 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:39.151692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19227126-720b-4548-ba05-fdbc4b572002 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:39.151966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19227126-720b-4548-ba05-fdbc4b572002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:39.196066 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-19227126-720b-4548-ba05-fdbc4b572002 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:39.197317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-19227126-720b-4548-ba05-fdbc4b572002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:39.224660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbed2d-9a06-4b6a-952e-e3bd61a6d7ab None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:39.262175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15194a4f-d7d6-4b74-8afb-0bc9c168ba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:39.263115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15194a4f-d7d6-4b74-8afb-0bc9c168ba72 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:39.263538 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15194a4f-d7d6-4b74-8afb-0bc9c168ba72 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:39.263951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15194a4f-d7d6-4b74-8afb-0bc9c168ba72 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:39.265132 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15194a4f-d7d6-4b74-8afb-0bc9c168ba72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:39.265571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15194a4f-d7d6-4b74-8afb-0bc9c168ba72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:39.267247 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15194a4f-d7d6-4b74-8afb-0bc9c168ba72 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:39.281642 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 14/28] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:38:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:38:39.282931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15194a4f-d7d6-4b74-8afb-0bc9c168ba72 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:39.353752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 15/29] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:38:39.355017 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19227126-720b-4548-ba05-fdbc4b572002 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:39.390848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:39.411852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15194a4f-d7d6-4b74-8afb-0bc9c168ba72 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:39.451563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a82c9-43ad-405c-bed9-33291b421bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:39.452621 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a82c9-43ad-405c-bed9-33291b421bb4 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:39.453074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a82c9-43ad-405c-bed9-33291b421bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:39.453554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a82c9-43ad-405c-bed9-33291b421bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:39.454529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a82c9-43ad-405c-bed9-33291b421bb4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:39.456254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a82c9-43ad-405c-bed9-33291b421bb4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:39.501306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:39.517192 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19227126-720b-4548-ba05-fdbc4b572002 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:39.557921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9addc-5c12-41f4-9006-3eb048e173e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:39.557921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9addc-5c12-41f4-9006-3eb048e173e4 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:39.557921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9addc-5c12-41f4-9006-3eb048e173e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:39.557921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9addc-5c12-41f4-9006-3eb048e173e4 None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:39.558291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9addc-5c12-41f4-9006-3eb048e173e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:39.558291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9addc-5c12-41f4-9006-3eb048e173e4 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:39.691652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 15/30] 10.222.0.121 () {60 vars in 1136 bytes} [Thu May 26 17:38:39 2022] POST /identity/v3/roles => generated 220 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:38:39.692719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a82c9-43ad-405c-bed9-33291b421bb4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:39.782319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 16/31] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:38:39 2022] POST /identity/v3/projects => generated 302 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:38:39.784556 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65ebe8-25e3-44dd-a3ac-7291ed5fcf72 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:39.785077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65ebe8-25e3-44dd-a3ac-7291ed5fcf72 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:39.785618 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65ebe8-25e3-44dd-a3ac-7291ed5fcf72 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:39.787307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 17/32] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:39 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:39.797315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893f0a3-3aed-43f7-baec-2ceeb76d391f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:39.797951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893f0a3-3aed-43f7-baec-2ceeb76d391f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:39.798437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893f0a3-3aed-43f7-baec-2ceeb76d391f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:39.812000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a82c9-43ad-405c-bed9-33291b421bb4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:39.828021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-7893f0a3-3aed-43f7-baec-2ceeb76d391f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:39.834882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7893f0a3-3aed-43f7-baec-2ceeb76d391f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:39.860863 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b28f26-3648-407e-afb8-708e68f63bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:39.861577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b28f26-3648-407e-afb8-708e68f63bd8 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:39.861912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b28f26-3648-407e-afb8-708e68f63bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:39.862243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b28f26-3648-407e-afb8-708e68f63bd8 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:39.863061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b28f26-3648-407e-afb8-708e68f63bd8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:39.864680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b28f26-3648-407e-afb8-708e68f63bd8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:39.872363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:39.959725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 18/33] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:39.998115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 16/34] 10.222.0.121 () {60 vars in 1136 bytes} [Thu May 26 17:38:39 2022] POST /identity/v3/roles => generated 226 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:38:40.000836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893f0a3-3aed-43f7-baec-2ceeb76d391f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:40.118032 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:40.164607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893f0a3-3aed-43f7-baec-2ceeb76d391f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:40.209010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84192e1a-d69c-44d1-8db7-9de29cecb4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:40.209914 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84192e1a-d69c-44d1-8db7-9de29cecb4c9 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:40.210188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84192e1a-d69c-44d1-8db7-9de29cecb4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:40.210476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84192e1a-d69c-44d1-8db7-9de29cecb4c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:40.215941 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55cf22-78d0-44a3-b87c-e7043dfd23b0 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:40.216494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55cf22-78d0-44a3-b87c-e7043dfd23b0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:40.216780 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55cf22-78d0-44a3-b87c-e7043dfd23b0 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:40.218153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 17/35] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:38:40.230575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbcf90-ec86-4eb0-97df-5d73b4740276 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:40.231224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbcf90-ec86-4eb0-97df-5d73b4740276 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:40.231722 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbcf90-ec86-4eb0-97df-5d73b4740276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:40.231822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84192e1a-d69c-44d1-8db7-9de29cecb4c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-84192e1a-d69c-44d1-8db7-9de29cecb4c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:40.239983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:40.240911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:38:40.242292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 19/36] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:38:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:38:40.253463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84192e1a-d69c-44d1-8db7-9de29cecb4c9 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:40.265041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e8bbcf90-ec86-4eb0-97df-5d73b4740276 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:40.267078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e8bbcf90-ec86-4eb0-97df-5d73b4740276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:40.365659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 18/37] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:38:40.368043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eca37-0057-4d85-9548-36a6d94ae677 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:40.368721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eca37-0057-4d85-9548-36a6d94ae677 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:40.369205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eca37-0057-4d85-9548-36a6d94ae677 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:40.371445 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 19/38] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:40 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:40.376843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165067a-b028-4bd3-ba43-7d05a95566bf None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:40.377132 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165067a-b028-4bd3-ba43-7d05a95566bf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:40.377420 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165067a-b028-4bd3-ba43-7d05a95566bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:40.396342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d165067a-b028-4bd3-ba43-7d05a95566bf None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:40.398088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d165067a-b028-4bd3-ba43-7d05a95566bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:40.401402 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84192e1a-d69c-44d1-8db7-9de29cecb4c9 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:40.429516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:40.437326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90e022-4c54-4a0d-ab3a-6c1ddd59e343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:40.438446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90e022-4c54-4a0d-ab3a-6c1ddd59e343 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:40.438446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90e022-4c54-4a0d-ab3a-6c1ddd59e343 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:40.438621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90e022-4c54-4a0d-ab3a-6c1ddd59e343 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:40.439338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90e022-4c54-4a0d-ab3a-6c1ddd59e343 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:40.439647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90e022-4c54-4a0d-ab3a-6c1ddd59e343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:40.441180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90e022-4c54-4a0d-ab3a-6c1ddd59e343 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:40.450293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 20/39] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:38:40 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:40.451378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90e022-4c54-4a0d-ab3a-6c1ddd59e343 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:40.468930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 20/40] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:38:40.469982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165067a-b028-4bd3-ba43-7d05a95566bf None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:40.527417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:40.560225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90e022-4c54-4a0d-ab3a-6c1ddd59e343 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:40.572484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165067a-b028-4bd3-ba43-7d05a95566bf None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:40.594676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5eeab-3416-4aac-81f4-4658d276561a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:40.595512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5eeab-3416-4aac-81f4-4658d276561a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:40.595799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5eeab-3416-4aac-81f4-4658d276561a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:40.596079 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5eeab-3416-4aac-81f4-4658d276561a None admin] PATH_INFO: `/v3/domains/default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:40.597273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5eeab-3416-4aac-81f4-4658d276561a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:40.598729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5eeab-3416-4aac-81f4-4658d276561a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:40.600378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 21/41] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:38:40 2022] GET /identity/v3/domains/default => generated 200 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:40.601181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5eeab-3416-4aac-81f4-4658d276561a None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:40.609025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad5175-fe65-4760-b6b8-afddf33c1dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:40.609909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad5175-fe65-4760-b6b8-afddf33c1dd0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:40.610194 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad5175-fe65-4760-b6b8-afddf33c1dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:40.610470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad5175-fe65-4760-b6b8-afddf33c1dd0 None admin] PATH_INFO: `/v3/users/9845c1dd09f34a0aa117c5e212e20304` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:40.611703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad5175-fe65-4760-b6b8-afddf33c1dd0 None admin] RBAC: Authorizing `identity:get_user(user_id=9845c1dd09f34a0aa117c5e212e20304)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:40.613160 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad5175-fe65-4760-b6b8-afddf33c1dd0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:40.614803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 21/42] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:40 2022] GET /identity/v3/users/9845c1dd09f34a0aa117c5e212e20304 => generated 248 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:40.614803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad5175-fe65-4760-b6b8-afddf33c1dd0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:40.665738 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:40.712559 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5eeab-3416-4aac-81f4-4658d276561a None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:40.724292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad5175-fe65-4760-b6b8-afddf33c1dd0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:40.748220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:40.751010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52d3bf-7a22-4535-a854-6a5cee4d1db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:40.751726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52d3bf-7a22-4535-a854-6a5cee4d1db2 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:40.752014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52d3bf-7a22-4535-a854-6a5cee4d1db2 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:40.752293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52d3bf-7a22-4535-a854-6a5cee4d1db2 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:40.753044 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52d3bf-7a22-4535-a854-6a5cee4d1db2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:40.754425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52d3bf-7a22-4535-a854-6a5cee4d1db2 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:40.759921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a266ba5-97cb-4356-9a16-1c2f6b41bfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:40.760586 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a266ba5-97cb-4356-9a16-1c2f6b41bfcf None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:40.760870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a266ba5-97cb-4356-9a16-1c2f6b41bfcf None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:40.761147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a266ba5-97cb-4356-9a16-1c2f6b41bfcf None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:40.761858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a266ba5-97cb-4356-9a16-1c2f6b41bfcf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:40.763352 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a266ba5-97cb-4356-9a16-1c2f6b41bfcf None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:40.874917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 22/43] 10.222.0.121 () {60 vars in 1136 bytes} [Thu May 26 17:38:40 2022] POST /identity/v3/roles => generated 224 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:38:40.874917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52d3bf-7a22-4535-a854-6a5cee4d1db2 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:40.884549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 22/44] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:38:40 2022] POST /identity/v3/projects => generated 313 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:38:40.992778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52d3bf-7a22-4535-a854-6a5cee4d1db2 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:41.014289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:41.034151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b00cf-5948-4139-8cd5-25b4a39f4b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:41.035036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b00cf-5948-4139-8cd5-25b4a39f4b3a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:41.035339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b00cf-5948-4139-8cd5-25b4a39f4b3a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:41.035660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b00cf-5948-4139-8cd5-25b4a39f4b3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:41.037105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b00cf-5948-4139-8cd5-25b4a39f4b3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:41.039149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b00cf-5948-4139-8cd5-25b4a39f4b3a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:41.041115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 23/45] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:38:40 2022] GET /identity/v3/domains/default => generated 200 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:41.047152 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a266ba5-97cb-4356-9a16-1c2f6b41bfcf None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:41.170718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a266ba5-97cb-4356-9a16-1c2f6b41bfcf None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:41.204563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f71e05-becd-4531-b9be-61ab6bb00265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:41.205442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f71e05-becd-4531-b9be-61ab6bb00265 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:41.205723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f71e05-becd-4531-b9be-61ab6bb00265 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:41.205999 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f71e05-becd-4531-b9be-61ab6bb00265 None admin] PATH_INFO: `/v3/domains/default/users/9845c1dd09f34a0aa117c5e212e20304/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:41.207999 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f71e05-becd-4531-b9be-61ab6bb00265 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9845c1dd09f34a0aa117c5e212e20304, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:41.209438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f71e05-becd-4531-b9be-61ab6bb00265 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:41.285368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 23/46] 10.222.0.121 () {58 vars in 1366 bytes} [Thu May 26 17:38:41 2022] PUT /identity/v3/domains/default/users/9845c1dd09f34a0aa117c5e212e20304/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:38:43.179535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546fa02-a1d9-4934-974e-48abfe6dfee8 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:43.180192 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546fa02-a1d9-4934-974e-48abfe6dfee8 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:43.180192 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546fa02-a1d9-4934-974e-48abfe6dfee8 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:43.181564 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 24/47] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:43.187073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf34c1-42fe-455b-ba63-583285afc6cd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:43.187528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf34c1-42fe-455b-ba63-583285afc6cd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:43.187776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf34c1-42fe-455b-ba63-583285afc6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:43.205815 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7cbf34c1-42fe-455b-ba63-583285afc6cd None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:43.207074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7cbf34c1-42fe-455b-ba63-583285afc6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:43.236979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:43.277213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 24/48] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:43.300964 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546fa02-a1d9-4934-974e-48abfe6dfee8 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:43.405926 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546fa02-a1d9-4934-974e-48abfe6dfee8 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:43.440125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fad87-56c4-4569-9772-65e7fd7df0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:43.440923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fad87-56c4-4569-9772-65e7fd7df0c3 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:43.441204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fad87-56c4-4569-9772-65e7fd7df0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:43.441479 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fad87-56c4-4569-9772-65e7fd7df0c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:43.442641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fad87-56c4-4569-9772-65e7fd7df0c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:43.444646 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fad87-56c4-4569-9772-65e7fd7df0c3 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:43.446409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 25/49] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:38:43 2022] GET /identity/v3/domains/default => generated 200 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:43.451878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf34c1-42fe-455b-ba63-583285afc6cd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:43.481228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75539ec-4be4-46b8-a220-00fd9cc7b4b8 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:43.481515 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75539ec-4be4-46b8-a220-00fd9cc7b4b8 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:43.481794 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75539ec-4be4-46b8-a220-00fd9cc7b4b8 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:43.483222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 26/50] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:38:43.488502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b0164-c92c-4540-8b0a-bb7334d00397 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:43.488787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b0164-c92c-4540-8b0a-bb7334d00397 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:43.489065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b0164-c92c-4540-8b0a-bb7334d00397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:43.505213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d64b0164-c92c-4540-8b0a-bb7334d00397 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:43.506404 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d64b0164-c92c-4540-8b0a-bb7334d00397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:43.555520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf34c1-42fe-455b-ba63-583285afc6cd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:43.557613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:43.575452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 27/51] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:38:43.590268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d73cb-8cc6-419e-859f-435633dbecbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:43.590965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d73cb-8cc6-419e-859f-435633dbecbd None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:43.591249 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d73cb-8cc6-419e-859f-435633dbecbd None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:43.591526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d73cb-8cc6-419e-859f-435633dbecbd None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:43.592231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d73cb-8cc6-419e-859f-435633dbecbd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:43.593743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d73cb-8cc6-419e-859f-435633dbecbd None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:43.598427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b0164-c92c-4540-8b0a-bb7334d00397 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:43.699520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b0164-c92c-4540-8b0a-bb7334d00397 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:43.722936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 25/52] 10.222.0.121 () {60 vars in 1145 bytes} [Thu May 26 17:38:43 2022] POST /identity/v3/projects => generated 299 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:38:43.740114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d5e76-ef18-4e0a-8c00-162ac6442de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:43.740996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d5e76-ef18-4e0a-8c00-162ac6442de3 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:43.741621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d5e76-ef18-4e0a-8c00-162ac6442de3 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:43.742045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d5e76-ef18-4e0a-8c00-162ac6442de3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:43.754506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d5e76-ef18-4e0a-8c00-162ac6442de3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d32d5e76-ef18-4e0a-8c00-162ac6442de3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:43.762978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:43.763937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:38:43.764676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 28/53] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:38:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:38:43.770297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d73cb-8cc6-419e-859f-435633dbecbd None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:43.794842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:43.888678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d73cb-8cc6-419e-859f-435633dbecbd None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:43.925184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc95fb2-3c6f-43d3-852a-c6e6e3d7a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:43.925869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc95fb2-3c6f-43d3-852a-c6e6e3d7a49c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:43.926156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc95fb2-3c6f-43d3-852a-c6e6e3d7a49c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:43.926435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc95fb2-3c6f-43d3-852a-c6e6e3d7a49c None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:43.927267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc95fb2-3c6f-43d3-852a-c6e6e3d7a49c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:43.927576 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc95fb2-3c6f-43d3-852a-c6e6e3d7a49c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:43.929504 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc95fb2-3c6f-43d3-852a-c6e6e3d7a49c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:43.941942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 26/54] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:38:43 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:43.948537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d5e76-ef18-4e0a-8c00-162ac6442de3 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:43.995934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:44.052235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d5e76-ef18-4e0a-8c00-162ac6442de3 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:44.089759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd9516-3bc8-422e-909f-cfd448ba42e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:44.090710 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd9516-3bc8-422e-909f-cfd448ba42e8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:44.091436 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd9516-3bc8-422e-909f-cfd448ba42e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:44.091723 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd9516-3bc8-422e-909f-cfd448ba42e8 None admin] PATH_INFO: `/v3/users/9845c1dd09f34a0aa117c5e212e20304` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:44.094883 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd9516-3bc8-422e-909f-cfd448ba42e8 None admin] RBAC: Authorizing `identity:get_user(user_id=9845c1dd09f34a0aa117c5e212e20304)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:44.096555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd9516-3bc8-422e-909f-cfd448ba42e8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:44.098309 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 29/55] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:43 2022] GET /identity/v3/users/9845c1dd09f34a0aa117c5e212e20304 => generated 248 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:44.104473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc95fb2-3c6f-43d3-852a-c6e6e3d7a49c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:44.190050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:44.208629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc95fb2-3c6f-43d3-852a-c6e6e3d7a49c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:44.244417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ee027-6978-483d-941b-cbe5bd9bd213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:44.245262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ee027-6978-483d-941b-cbe5bd9bd213 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:44.245543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ee027-6978-483d-941b-cbe5bd9bd213 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:44.245817 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ee027-6978-483d-941b-cbe5bd9bd213 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:44.247075 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ee027-6978-483d-941b-cbe5bd9bd213 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:44.249338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ee027-6978-483d-941b-cbe5bd9bd213 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:44.252260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 27/56] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:38:44 2022] GET /identity/v3/domains/default => generated 200 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:44.258452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd9516-3bc8-422e-909f-cfd448ba42e8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:44.365295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd9516-3bc8-422e-909f-cfd448ba42e8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:44.378937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:44.401287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3be822-a8db-424c-a32f-51429cbe7231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:44.402545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3be822-a8db-424c-a32f-51429cbe7231 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:44.402545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3be822-a8db-424c-a32f-51429cbe7231 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:44.402545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3be822-a8db-424c-a32f-51429cbe7231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:44.432259 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3be822-a8db-424c-a32f-51429cbe7231 None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.domain.id=default, user.id=9845c1dd09f34a0aa117c5e212e20304) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:44.432259 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3be822-a8db-424c-a32f-51429cbe7231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:44.432259 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3be822-a8db-424c-a32f-51429cbe7231 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:44.432259 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 30/57] 10.222.0.121 () {56 vars in 1327 bytes} [Thu May 26 17:38:44 2022] GET /identity/v3/role_assignments?user.id=9845c1dd09f34a0aa117c5e212e20304&scope.domain.id=default&role.id=4f574961798246a4b12ad2276e67685d => generated 532 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:45.977696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e288b-06e9-49de-b4c3-9c4877d5aaa6 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:45.978220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e288b-06e9-49de-b4c3-9c4877d5aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:45.978307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e288b-06e9-49de-b4c3-9c4877d5aaa6 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:45.979760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 28/58] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:45.985021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05c7ce-f76c-4ad5-8928-420a07066a52 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:45.985430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05c7ce-f76c-4ad5-8928-420a07066a52 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:45.985590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05c7ce-f76c-4ad5-8928-420a07066a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:46.003304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-9c05c7ce-f76c-4ad5-8928-420a07066a52 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:46.004501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9c05c7ce-f76c-4ad5-8928-420a07066a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:46.071500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 31/59] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:46.096053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e288b-06e9-49de-b4c3-9c4877d5aaa6 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:46.198071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e288b-06e9-49de-b4c3-9c4877d5aaa6 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:46.231559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3d990-b93f-4c54-940b-ee8edb8a7898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:46.232280 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d990-b93f-4c54-940b-ee8edb8a7898 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:46.232562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d990-b93f-4c54-940b-ee8edb8a7898 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:46.232838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d990-b93f-4c54-940b-ee8edb8a7898 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:46.234003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3d990-b93f-4c54-940b-ee8edb8a7898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:46.235480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3d990-b93f-4c54-940b-ee8edb8a7898 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:46.237073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 29/60] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:38:46 2022] GET /identity/v3/domains/default => generated 200 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:46.242805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05c7ce-f76c-4ad5-8928-420a07066a52 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:46.337895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05c7ce-f76c-4ad5-8928-420a07066a52 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:46.345059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:46.371899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59531cf-50fc-422f-89c0-5a97a4562590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:46.372578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59531cf-50fc-422f-89c0-5a97a4562590 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:46.372864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59531cf-50fc-422f-89c0-5a97a4562590 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:46.373141 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59531cf-50fc-422f-89c0-5a97a4562590 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:46.373857 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59531cf-50fc-422f-89c0-5a97a4562590 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:46.375393 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59531cf-50fc-422f-89c0-5a97a4562590 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:46.458155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 32/61] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:38:46 2022] POST /identity/v3/users => generated 276 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:38:49.793151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b37c7-f051-49d4-bc1e-b0a31b54b14a None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:49.794185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b37c7-f051-49d4-bc1e-b0a31b54b14a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:49.794768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b37c7-f051-49d4-bc1e-b0a31b54b14a None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:49.796651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 30/62] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:49 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:49.801831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03be6a-99e1-4c5e-a36b-7459b92c2131 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:49.802665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03be6a-99e1-4c5e-a36b-7459b92c2131 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:49.803163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03be6a-99e1-4c5e-a36b-7459b92c2131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:49.822071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-0f03be6a-99e1-4c5e-a36b-7459b92c2131 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:49.827740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0f03be6a-99e1-4c5e-a36b-7459b92c2131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:49.913296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 33/63] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:49.937165 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b37c7-f051-49d4-bc1e-b0a31b54b14a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:49.960481 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:50.084189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b37c7-f051-49d4-bc1e-b0a31b54b14a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:50.138571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3d675-df96-4efd-9111-72ce14391e3f None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.138887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3d675-df96-4efd-9111-72ce14391e3f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.139160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3d675-df96-4efd-9111-72ce14391e3f None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.140532 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 34/64] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:38:50.145948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e47c1-bcb1-4f1c-b33d-4667a1e60bdc None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.146242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e47c1-bcb1-4f1c-b33d-4667a1e60bdc None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.146526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e47c1-bcb1-4f1c-b33d-4667a1e60bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.159680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822851e2-d16f-4b63-916c-8e6fadaae3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:50.160500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822851e2-d16f-4b63-916c-8e6fadaae3a0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.160784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822851e2-d16f-4b63-916c-8e6fadaae3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.161060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822851e2-d16f-4b63-916c-8e6fadaae3a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.175750 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-8e9e47c1-bcb1-4f1c-b33d-4667a1e60bdc None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:50.176951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-8e9e47c1-bcb1-4f1c-b33d-4667a1e60bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:50.191823 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822851e2-d16f-4b63-916c-8e6fadaae3a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-822851e2-d16f-4b63-916c-8e6fadaae3a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:50.201522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:50.202405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:38:50.203602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 31/65] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:38:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:38:50.213028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822851e2-d16f-4b63-916c-8e6fadaae3a0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:50.273312 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 35/66] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:50.282795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94496b0-0395-44e8-a695-d71f5aded6d1 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.283363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94496b0-0395-44e8-a695-d71f5aded6d1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.283959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94496b0-0395-44e8-a695-d71f5aded6d1 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.285552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 36/67] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:50 2022] GET /identity => generated 271 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:50.293734 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79706057-529c-48a8-a13a-b89a95a774b1 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.294022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79706057-529c-48a8-a13a-b89a95a774b1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.294301 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79706057-529c-48a8-a13a-b89a95a774b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.314534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-79706057-529c-48a8-a13a-b89a95a774b1 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:50.315778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-79706057-529c-48a8-a13a-b89a95a774b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:50.318180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:50.333573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:50.344778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822851e2-d16f-4b63-916c-8e6fadaae3a0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:50.382303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba693cc-772b-45f7-a9f0-a938ef8408ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:50.383109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba693cc-772b-45f7-a9f0-a938ef8408ff None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.383383 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba693cc-772b-45f7-a9f0-a938ef8408ff None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.383964 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba693cc-772b-45f7-a9f0-a938ef8408ff None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.384429 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba693cc-772b-45f7-a9f0-a938ef8408ff None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:50.384748 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba693cc-772b-45f7-a9f0-a938ef8408ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:50.386150 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba693cc-772b-45f7-a9f0-a938ef8408ff None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:50.388363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 37/68] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:38:50.395132 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 32/69] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:38:50 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:50.396303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba693cc-772b-45f7-a9f0-a938ef8408ff None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:50.396667 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545424c0-6f4b-44bc-90bb-e51c55094c5e None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.396963 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545424c0-6f4b-44bc-90bb-e51c55094c5e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.398241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545424c0-6f4b-44bc-90bb-e51c55094c5e None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.404158 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 38/70] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:50 2022] GET /identity => generated 271 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:50.404158 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545424c0-6f4b-44bc-90bb-e51c55094c5e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:50.505040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba693cc-772b-45f7-a9f0-a938ef8408ff None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:50.519584 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545424c0-6f4b-44bc-90bb-e51c55094c5e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:50.545138 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ccb06-5103-4e13-b5c3-a536f67a8e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:50.545941 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ccb06-5103-4e13-b5c3-a536f67a8e97 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.546228 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ccb06-5103-4e13-b5c3-a536f67a8e97 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.546512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ccb06-5103-4e13-b5c3-a536f67a8e97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.554255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:50.558739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ccb06-5103-4e13-b5c3-a536f67a8e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:50.559492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d5fd1-5d30-4c6c-aa21-df2674dfb902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:50.560404 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d5fd1-5d30-4c6c-aa21-df2674dfb902 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.560694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d5fd1-5d30-4c6c-aa21-df2674dfb902 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.560976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d5fd1-5d30-4c6c-aa21-df2674dfb902 None admin] PATH_INFO: `/v3/users/7201ba55061e4bf4882217b64adb67b6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f24ccb06-5103-4e13-b5c3-a536f67a8e97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:50.566659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:50.567870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:38:50.568981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 33/71] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:38:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:38:50.573244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb56cc2-c6d9-4727-b058-a8266396ac16 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.573545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb56cc2-c6d9-4727-b058-a8266396ac16 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.573836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb56cc2-c6d9-4727-b058-a8266396ac16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.580871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d5fd1-5d30-4c6c-aa21-df2674dfb902 None admin] RBAC: Authorizing `identity:get_user(user_id=7201ba55061e4bf4882217b64adb67b6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:50.582376 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d5fd1-5d30-4c6c-aa21-df2674dfb902 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:50.589846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 39/72] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:50 2022] GET /identity/v3/users/7201ba55061e4bf4882217b64adb67b6 => generated 276 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:50.589846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d5fd1-5d30-4c6c-aa21-df2674dfb902 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:50.593590 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1cb56cc2-c6d9-4727-b058-a8266396ac16 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:50.594925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1cb56cc2-c6d9-4727-b058-a8266396ac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:50.604310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:50.667883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 34/73] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:50.680130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb56cc2-c6d9-4727-b058-a8266396ac16 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:50.711000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d5fd1-5d30-4c6c-aa21-df2674dfb902 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:50.753741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9923d-6676-4373-882a-e7dbb1d884e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:50.754794 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9923d-6676-4373-882a-e7dbb1d884e7 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.755241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9923d-6676-4373-882a-e7dbb1d884e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.755653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9923d-6676-4373-882a-e7dbb1d884e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.767288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9923d-6676-4373-882a-e7dbb1d884e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-f0e9923d-6676-4373-882a-e7dbb1d884e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:50.776137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:50.778238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 40/74] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:38:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:38:50.782299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9923d-6676-4373-882a-e7dbb1d884e7 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:50.794726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb56cc2-c6d9-4727-b058-a8266396ac16 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:50.822642 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:50.831581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e436d9-80e1-4730-a1e7-105f38ec5de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:50.833687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e436d9-80e1-4730-a1e7-105f38ec5de9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.833687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e436d9-80e1-4730-a1e7-105f38ec5de9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.833687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e436d9-80e1-4730-a1e7-105f38ec5de9 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.833687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e436d9-80e1-4730-a1e7-105f38ec5de9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:50.833987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e436d9-80e1-4730-a1e7-105f38ec5de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:50.835749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e436d9-80e1-4730-a1e7-105f38ec5de9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:50.843970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 35/75] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:38:50 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:50.844654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e436d9-80e1-4730-a1e7-105f38ec5de9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:50.854472 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:50.886259 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9923d-6676-4373-882a-e7dbb1d884e7 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:50.920599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2cc94e-4956-4af6-8722-173e5a5c92f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:50.921259 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2cc94e-4956-4af6-8722-173e5a5c92f2 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.921545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2cc94e-4956-4af6-8722-173e5a5c92f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.921824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2cc94e-4956-4af6-8722-173e5a5c92f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.947374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e436d9-80e1-4730-a1e7-105f38ec5de9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:50.982716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c32a6a-54a1-41a8-9cc0-b29650d3b600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:50.983586 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c32a6a-54a1-41a8-9cc0-b29650d3b600 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:50.983872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c32a6a-54a1-41a8-9cc0-b29650d3b600 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:50.984154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c32a6a-54a1-41a8-9cc0-b29650d3b600 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:50.992971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2cc94e-4956-4af6-8722-173e5a5c92f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:50.994448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2cc94e-4956-4af6-8722-173e5a5c92f2 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:50.994998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c32a6a-54a1-41a8-9cc0-b29650d3b600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-31c32a6a-54a1-41a8-9cc0-b29650d3b600 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:51.002666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:51.003563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:38:51.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 36/76] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:38:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:38:51.005124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe62073-5294-41f7-993c-6502a7e53a06 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:51.005411 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe62073-5294-41f7-993c-6502a7e53a06 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:51.005697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe62073-5294-41f7-993c-6502a7e53a06 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:51.007134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 37/77] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:51.008007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe62073-5294-41f7-993c-6502a7e53a06 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:51.063299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:51.071774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:51.104223 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 41/78] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:38:50 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:38:51.105374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2cc94e-4956-4af6-8722-173e5a5c92f2 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:51.121826 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe62073-5294-41f7-993c-6502a7e53a06 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:51.161676 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0350b7-05aa-47b3-854a-bba2eb6a74ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:51.162365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0350b7-05aa-47b3-854a-bba2eb6a74ac None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:51.162644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0350b7-05aa-47b3-854a-bba2eb6a74ac None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:51.163507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0350b7-05aa-47b3-854a-bba2eb6a74ac None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:51.164296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0350b7-05aa-47b3-854a-bba2eb6a74ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:51.164599 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0350b7-05aa-47b3-854a-bba2eb6a74ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:51.165986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0350b7-05aa-47b3-854a-bba2eb6a74ac None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:51.174390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 38/79] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:38:51 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:51.175274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0350b7-05aa-47b3-854a-bba2eb6a74ac None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:51.220373 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2cc94e-4956-4af6-8722-173e5a5c92f2 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:51.253994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153dea0-df9f-46fe-ae29-9b4e84673f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:51.254899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153dea0-df9f-46fe-ae29-9b4e84673f62 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:51.255185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153dea0-df9f-46fe-ae29-9b4e84673f62 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:51.255458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153dea0-df9f-46fe-ae29-9b4e84673f62 None admin] PATH_INFO: `/v3/users/9845c1dd09f34a0aa117c5e212e20304` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:51.256725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153dea0-df9f-46fe-ae29-9b4e84673f62 None admin] RBAC: Authorizing `identity:get_user(user_id=9845c1dd09f34a0aa117c5e212e20304)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:51.258213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153dea0-df9f-46fe-ae29-9b4e84673f62 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:51.259888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 42/80] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:51 2022] GET /identity/v3/users/9845c1dd09f34a0aa117c5e212e20304 => generated 248 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:51.261065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252254c-5674-465b-a8a7-907a52a3b610 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:51.261382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252254c-5674-465b-a8a7-907a52a3b610 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:51.261619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252254c-5674-465b-a8a7-907a52a3b610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:51.279051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-e252254c-5674-465b-a8a7-907a52a3b610 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:51.280250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e252254c-5674-465b-a8a7-907a52a3b610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:51.282493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0350b7-05aa-47b3-854a-bba2eb6a74ac None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:51.296141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:51.310488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:51.317018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e042d0-32d7-456f-a47f-5f245bef0d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:51.317690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e042d0-32d7-456f-a47f-5f245bef0d14 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:51.317971 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e042d0-32d7-456f-a47f-5f245bef0d14 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:51.318245 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e042d0-32d7-456f-a47f-5f245bef0d14 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:51.319079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e042d0-32d7-456f-a47f-5f245bef0d14 None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:51.319364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e042d0-32d7-456f-a47f-5f245bef0d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:51.320743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e042d0-32d7-456f-a47f-5f245bef0d14 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:51.329329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 39/81] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:38:51 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:51.330146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e042d0-32d7-456f-a47f-5f245bef0d14 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:51.356146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 43/82] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:51.356812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252254c-5674-465b-a8a7-907a52a3b610 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:51.453595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e042d0-32d7-456f-a47f-5f245bef0d14 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:51.478933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252254c-5674-465b-a8a7-907a52a3b610 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:51.490460 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca994468-ef0d-4ebe-9e01-0775050c3886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:51.491443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca994468-ef0d-4ebe-9e01-0775050c3886 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:51.491721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca994468-ef0d-4ebe-9e01-0775050c3886 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:51.492001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca994468-ef0d-4ebe-9e01-0775050c3886 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:51.493232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca994468-ef0d-4ebe-9e01-0775050c3886 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:51.494708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca994468-ef0d-4ebe-9e01-0775050c3886 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:51.496528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 40/83] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:38:51 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:51.497541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca994468-ef0d-4ebe-9e01-0775050c3886 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:51.521336 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc70e98-fe69-4dd8-8b2a-51a1581dd759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:51.522795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc70e98-fe69-4dd8-8b2a-51a1581dd759 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:51.523112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc70e98-fe69-4dd8-8b2a-51a1581dd759 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:51.523843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc70e98-fe69-4dd8-8b2a-51a1581dd759 None admin] PATH_INFO: `/v3/users/7201ba55061e4bf4882217b64adb67b6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:51.524847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc70e98-fe69-4dd8-8b2a-51a1581dd759 None admin] RBAC: Authorizing `identity:get_user(user_id=7201ba55061e4bf4882217b64adb67b6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:51.527061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc70e98-fe69-4dd8-8b2a-51a1581dd759 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:51.534194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 44/84] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:51 2022] GET /identity/v3/users/7201ba55061e4bf4882217b64adb67b6 => generated 276 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:51.534194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc70e98-fe69-4dd8-8b2a-51a1581dd759 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:51.549674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:51.572945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:51.623255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca994468-ef0d-4ebe-9e01-0775050c3886 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:51.645499 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc70e98-fe69-4dd8-8b2a-51a1581dd759 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:51.661169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bb7c-6058-4db8-aa16-00e47ffa5183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:51.661836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bb7c-6058-4db8-aa16-00e47ffa5183 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:51.662122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bb7c-6058-4db8-aa16-00e47ffa5183 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:51.662398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bb7c-6058-4db8-aa16-00e47ffa5183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:51.690614 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e5f7f-cf85-4ae0-8945-647ab8eeaa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:51.691147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e5f7f-cf85-4ae0-8945-647ab8eeaa51 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:51.691430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e5f7f-cf85-4ae0-8945-647ab8eeaa51 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:51.691713 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e5f7f-cf85-4ae0-8945-647ab8eeaa51 None admin] PATH_INFO: `/v3/users/7201ba55061e4bf4882217b64adb67b6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:51.692906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e5f7f-cf85-4ae0-8945-647ab8eeaa51 None admin] RBAC: Authorizing `identity:get_user(user_id=7201ba55061e4bf4882217b64adb67b6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:51.694900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e5f7f-cf85-4ae0-8945-647ab8eeaa51 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:51.696667 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 45/85] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:51 2022] GET /identity/v3/users/7201ba55061e4bf4882217b64adb67b6 => generated 276 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:51.697457 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e5f7f-cf85-4ae0-8945-647ab8eeaa51 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:51.735077 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86bb7c-6058-4db8-aa16-00e47ffa5183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:51.736575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86bb7c-6058-4db8-aa16-00e47ffa5183 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:51.778903 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:51.808641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e5f7f-cf85-4ae0-8945-647ab8eeaa51 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:51.811990 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:51.841698 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 41/86] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:38:51 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:38:51.842562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bb7c-6058-4db8-aa16-00e47ffa5183 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:51.849285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94acd0-68c8-45e2-95fd-d99d181c9211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:51.850096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94acd0-68c8-45e2-95fd-d99d181c9211 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:51.850420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94acd0-68c8-45e2-95fd-d99d181c9211 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:51.850687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94acd0-68c8-45e2-95fd-d99d181c9211 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:51.852371 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94acd0-68c8-45e2-95fd-d99d181c9211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:51.853814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94acd0-68c8-45e2-95fd-d99d181c9211 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:51.855460 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 46/87] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:38:51 2022] GET /identity/v3/domains/default => generated 200 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:51.856239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94acd0-68c8-45e2-95fd-d99d181c9211 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:51.944198 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bb7c-6058-4db8-aa16-00e47ffa5183 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:51.961351 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94acd0-68c8-45e2-95fd-d99d181c9211 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:51.981259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df96cf-6e3e-40a8-8079-91a391fa64b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:51.981926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df96cf-6e3e-40a8-8079-91a391fa64b6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:51.982206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df96cf-6e3e-40a8-8079-91a391fa64b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:51.982478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df96cf-6e3e-40a8-8079-91a391fa64b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:51.983337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df96cf-6e3e-40a8-8079-91a391fa64b6 None admin] RBAC: Adding query filter params (role.id=e87c07d8d70d46d89a2c4f7246edeff6, scope.project.id=8611e940b0fa4f8d910441467b31bdce, user.id=7201ba55061e4bf4882217b64adb67b6) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:51.983642 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df96cf-6e3e-40a8-8079-91a391fa64b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:51.985157 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df96cf-6e3e-40a8-8079-91a391fa64b6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:51.992281 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 42/88] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:38:51 2022] GET /identity/v3/role_assignments?user.id=7201ba55061e4bf4882217b64adb67b6&scope.project.id=8611e940b0fa4f8d910441467b31bdce&role.id=e87c07d8d70d46d89a2c4f7246edeff6 => generated 260 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:51.992917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df96cf-6e3e-40a8-8079-91a391fa64b6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:51.997171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf074b88-9b7f-4a32-a206-2b21622b00f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:51.997851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf074b88-9b7f-4a32-a206-2b21622b00f7 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:51.998140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf074b88-9b7f-4a32-a206-2b21622b00f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:51.998418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf074b88-9b7f-4a32-a206-2b21622b00f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:52.018652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:52.057800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:52.106459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf074b88-9b7f-4a32-a206-2b21622b00f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:52.109981 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf074b88-9b7f-4a32-a206-2b21622b00f7 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:52.153226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df96cf-6e3e-40a8-8079-91a391fa64b6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:52.207051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30424014-0215-4807-8fee-85fa779089f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:52.207548 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30424014-0215-4807-8fee-85fa779089f0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:52.207833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30424014-0215-4807-8fee-85fa779089f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:52.208108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30424014-0215-4807-8fee-85fa779089f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:52.243852 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 47/89] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:38:51 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:38:52.249013 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf074b88-9b7f-4a32-a206-2b21622b00f7 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:52.295122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30424014-0215-4807-8fee-85fa779089f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:52.296782 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30424014-0215-4807-8fee-85fa779089f0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:52.307005 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:52.330937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:52.364541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf074b88-9b7f-4a32-a206-2b21622b00f7 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:52.405271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec2d49-f174-46e5-ada2-3362fbda4519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:52.406155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec2d49-f174-46e5-ada2-3362fbda4519 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:52.406436 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec2d49-f174-46e5-ada2-3362fbda4519 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:52.406715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec2d49-f174-46e5-ada2-3362fbda4519 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:52.408004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec2d49-f174-46e5-ada2-3362fbda4519 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:52.409480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec2d49-f174-46e5-ada2-3362fbda4519 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:52.412056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 48/90] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:38:52 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:52.412056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec2d49-f174-46e5-ada2-3362fbda4519 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:52.440204 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 43/91] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:38:51 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:38:52.440204 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30424014-0215-4807-8fee-85fa779089f0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:52.559400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec2d49-f174-46e5-ada2-3362fbda4519 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:52.579893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30424014-0215-4807-8fee-85fa779089f0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:52.590038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:52.599308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e044f-f8f6-44f2-bb1f-4fe239bcd3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:52.599999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e044f-f8f6-44f2-bb1f-4fe239bcd3e8 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:52.604677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e044f-f8f6-44f2-bb1f-4fe239bcd3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:52.604677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e044f-f8f6-44f2-bb1f-4fe239bcd3e8 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:52.604677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e044f-f8f6-44f2-bb1f-4fe239bcd3e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:52.604677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e044f-f8f6-44f2-bb1f-4fe239bcd3e8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:52.613968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:52.619647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bd4db-db91-466d-9399-6abd41145f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:52.620615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bd4db-db91-466d-9399-6abd41145f46 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:52.620968 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bd4db-db91-466d-9399-6abd41145f46 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:52.621311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bd4db-db91-466d-9399-6abd41145f46 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:52.622610 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bd4db-db91-466d-9399-6abd41145f46 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:52.624449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bd4db-db91-466d-9399-6abd41145f46 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:52.626850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 44/92] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:38:52 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:52.627537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bd4db-db91-466d-9399-6abd41145f46 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:52.681837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 49/93] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:38:52 2022] POST /identity/v3/users => generated 290 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:38:52.692401 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e044f-f8f6-44f2-bb1f-4fe239bcd3e8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:52.742200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bd4db-db91-466d-9399-6abd41145f46 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:52.786901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a0705-ed4d-47a8-9be0-1895ebd67159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:52.788796 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a0705-ed4d-47a8-9be0-1895ebd67159 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:52.789683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a0705-ed4d-47a8-9be0-1895ebd67159 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:52.790159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a0705-ed4d-47a8-9be0-1895ebd67159 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:52.791450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a0705-ed4d-47a8-9be0-1895ebd67159 None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=8611e940b0fa4f8d910441467b31bdce, user.id=9845c1dd09f34a0aa117c5e212e20304) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:52.792030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a0705-ed4d-47a8-9be0-1895ebd67159 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:52.794111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a0705-ed4d-47a8-9be0-1895ebd67159 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:52.801474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 45/94] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:38:52 2022] GET /identity/v3/role_assignments?user.id=9845c1dd09f34a0aa117c5e212e20304&scope.project.id=8611e940b0fa4f8d910441467b31bdce&role.id=4f574961798246a4b12ad2276e67685d => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:52.802657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a0705-ed4d-47a8-9be0-1895ebd67159 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:52.857450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e044f-f8f6-44f2-bb1f-4fe239bcd3e8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:52.868402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:52.907761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced17ad-1f3e-416d-8a87-95dea66bf233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:52.908695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced17ad-1f3e-416d-8a87-95dea66bf233 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:52.908987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced17ad-1f3e-416d-8a87-95dea66bf233 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:52.909271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced17ad-1f3e-416d-8a87-95dea66bf233 None admin] PATH_INFO: `/v3/projects/5480346b7e284479ab3ab16d176d570f` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:52.910795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced17ad-1f3e-416d-8a87-95dea66bf233 None admin] RBAC: Authorizing `identity:get_project(project_id=5480346b7e284479ab3ab16d176d570f)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:52.912212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced17ad-1f3e-416d-8a87-95dea66bf233 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:52.915238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 50/95] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:38:52 2022] GET /identity/v3/projects/5480346b7e284479ab3ab16d176d570f => generated 313 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:52.926155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced17ad-1f3e-416d-8a87-95dea66bf233 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:52.942102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a0705-ed4d-47a8-9be0-1895ebd67159 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:52.986250 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41160144-1473-4334-b39b-d6a8ce750b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:52.987330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41160144-1473-4334-b39b-d6a8ce750b88 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:52.987839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41160144-1473-4334-b39b-d6a8ce750b88 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:52.988309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41160144-1473-4334-b39b-d6a8ce750b88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:52.990746 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41160144-1473-4334-b39b-d6a8ce750b88 None admin] RBAC: Adding query filter params (role.id=582f59fc05ce474287c6fdc503f9ac82, scope.project.id=8611e940b0fa4f8d910441467b31bdce, user.id=7201ba55061e4bf4882217b64adb67b6) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:52.992187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41160144-1473-4334-b39b-d6a8ce750b88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:52.994032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41160144-1473-4334-b39b-d6a8ce750b88 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:53.001099 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 46/96] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:38:52 2022] GET /identity/v3/role_assignments?user.id=7201ba55061e4bf4882217b64adb67b6&scope.project.id=8611e940b0fa4f8d910441467b31bdce&role.id=582f59fc05ce474287c6fdc503f9ac82 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:53.021889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:53.067827 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced17ad-1f3e-416d-8a87-95dea66bf233 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:53.118434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c8163-5f80-4901-be13-e6d7459df276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:53.119627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c8163-5f80-4901-be13-e6d7459df276 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:53.119926 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c8163-5f80-4901-be13-e6d7459df276 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:53.120213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c8163-5f80-4901-be13-e6d7459df276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:53.121123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c8163-5f80-4901-be13-e6d7459df276 None admin] RBAC: Adding query filter params (role.id=e87c07d8d70d46d89a2c4f7246edeff6, scope.project.id=5480346b7e284479ab3ab16d176d570f, user.id=7201ba55061e4bf4882217b64adb67b6) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:53.121446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c8163-5f80-4901-be13-e6d7459df276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:53.123010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c8163-5f80-4901-be13-e6d7459df276 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:53.133745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 51/97] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:38:52 2022] GET /identity/v3/role_assignments?user.id=7201ba55061e4bf4882217b64adb67b6&scope.project.id=5480346b7e284479ab3ab16d176d570f&role.id=e87c07d8d70d46d89a2c4f7246edeff6 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:38:54.671006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861ced2-d7cb-4516-b841-84c591ef3456 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:54.672140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861ced2-d7cb-4516-b841-84c591ef3456 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:54.672637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861ced2-d7cb-4516-b841-84c591ef3456 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:54.674529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 47/98] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:54 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:54.679995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef0e05-8009-4437-b990-0b170956980f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:54.680846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef0e05-8009-4437-b990-0b170956980f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:54.681335 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef0e05-8009-4437-b990-0b170956980f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:54.705263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-4aef0e05-8009-4437-b990-0b170956980f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:54.706886 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4aef0e05-8009-4437-b990-0b170956980f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:54.795824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 52/99] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:54.823806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861ced2-d7cb-4516-b841-84c591ef3456 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:54.950721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861ced2-d7cb-4516-b841-84c591ef3456 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:54.952729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:54.995848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6aaf3-8810-4254-86e2-05b3ea4b359a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:54.996678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6aaf3-8810-4254-86e2-05b3ea4b359a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:54.996966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6aaf3-8810-4254-86e2-05b3ea4b359a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:54.997247 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6aaf3-8810-4254-86e2-05b3ea4b359a None admin] PATH_INFO: `/v3/roles/member` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:55.011854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6aaf3-8810-4254-86e2-05b3ea4b359a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-54e6aaf3-8810-4254-86e2-05b3ea4b359a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:55.019690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:55.020609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:38:55.021384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 48/100] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:38:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:38:55.029003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef0e05-8009-4437-b990-0b170956980f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:55.125641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:55.188507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef0e05-8009-4437-b990-0b170956980f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:55.243623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8e647-d2b1-453c-ac2d-168c59a0bdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:55.244291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8e647-d2b1-453c-ac2d-168c59a0bdc4 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:55.244579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8e647-d2b1-453c-ac2d-168c59a0bdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:55.244907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8e647-d2b1-453c-ac2d-168c59a0bdc4 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:55.245670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8e647-d2b1-453c-ac2d-168c59a0bdc4 None admin] RBAC: Adding query filter params (name=member) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:55.245987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8e647-d2b1-453c-ac2d-168c59a0bdc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:55.247418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8e647-d2b1-453c-ac2d-168c59a0bdc4 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:55.263272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 53/101] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:38:55 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:55.269324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6aaf3-8810-4254-86e2-05b3ea4b359a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:55.405805 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6aaf3-8810-4254-86e2-05b3ea4b359a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:55.449937 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec647e-7302-4b2a-8ae7-4cbfe132fd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:55.450929 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec647e-7302-4b2a-8ae7-4cbfe132fd62 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:55.451215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec647e-7302-4b2a-8ae7-4cbfe132fd62 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:55.451495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec647e-7302-4b2a-8ae7-4cbfe132fd62 None admin] PATH_INFO: `/v3/users/7201ba55061e4bf4882217b64adb67b6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:55.452719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec647e-7302-4b2a-8ae7-4cbfe132fd62 None admin] RBAC: Authorizing `identity:get_user(user_id=7201ba55061e4bf4882217b64adb67b6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:55.454175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec647e-7302-4b2a-8ae7-4cbfe132fd62 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:55.456159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 49/102] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:55 2022] GET /identity/v3/users/7201ba55061e4bf4882217b64adb67b6 => generated 276 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:55.463934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8e647-d2b1-453c-ac2d-168c59a0bdc4 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:55.624672 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8e647-d2b1-453c-ac2d-168c59a0bdc4 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:55.649649 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:55.668953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487b0e1-3572-4118-a728-8ee6990b136d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:55.673011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487b0e1-3572-4118-a728-8ee6990b136d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:55.673011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487b0e1-3572-4118-a728-8ee6990b136d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:55.673011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487b0e1-3572-4118-a728-8ee6990b136d None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:55.673011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487b0e1-3572-4118-a728-8ee6990b136d None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:55.673435 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487b0e1-3572-4118-a728-8ee6990b136d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:55.675108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 54/103] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:38:55 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:55.681780 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec647e-7302-4b2a-8ae7-4cbfe132fd62 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:55.733274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:55.858251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec647e-7302-4b2a-8ae7-4cbfe132fd62 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:55.910486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d20db-6503-4384-b334-0763ac0ba711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:55.911408 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d20db-6503-4384-b334-0763ac0ba711 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:55.911694 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d20db-6503-4384-b334-0763ac0ba711 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:55.911981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d20db-6503-4384-b334-0763ac0ba711 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce/users/7201ba55061e4bf4882217b64adb67b6/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:55.914594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d20db-6503-4384-b334-0763ac0ba711 None admin] RBAC: Authorizing `identity:create_grant(project_id=8611e940b0fa4f8d910441467b31bdce, user_id=7201ba55061e4bf4882217b64adb67b6, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:55.916137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d20db-6503-4384-b334-0763ac0ba711 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:55.957777 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1204c-0c00-42c7-af41-b1810b27aa21 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:55.958312 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1204c-0c00-42c7-af41-b1810b27aa21 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:55.958386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1204c-0c00-42c7-af41-b1810b27aa21 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:55.959913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 55/104] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:38:55.965965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fe550-49c0-40bf-b8f1-eb6575e04e22 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:55.966279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fe550-49c0-40bf-b8f1-eb6575e04e22 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:55.966593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fe550-49c0-40bf-b8f1-eb6575e04e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:55.987418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 50/105] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:38:55 2022] PUT /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce/users/7201ba55061e4bf4882217b64adb67b6/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:38:55.992199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-df8fe550-49c0-40bf-b8f1-eb6575e04e22 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:55.992199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-df8fe550-49c0-40bf-b8f1-eb6575e04e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:56.008331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820e143-deaf-44fc-9552-84754349d10e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:56.009215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820e143-deaf-44fc-9552-84754349d10e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:56.009497 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820e143-deaf-44fc-9552-84754349d10e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:56.011408 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 51/106] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:56 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:38:56.017654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff9d66-ebe2-4e4f-b261-a97a5e2fc56f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:56.017945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff9d66-ebe2-4e4f-b261-a97a5e2fc56f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:56.020882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff9d66-ebe2-4e4f-b261-a97a5e2fc56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:56.049901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-57ff9d66-ebe2-4e4f-b261-a97a5e2fc56f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:56.051233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-57ff9d66-ebe2-4e4f-b261-a97a5e2fc56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:56.097732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 56/107] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:55 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:56.144925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fe550-49c0-40bf-b8f1-eb6575e04e22 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:56.185264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 52/108] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:38:56.231938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff9d66-ebe2-4e4f-b261-a97a5e2fc56f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:56.288495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fe550-49c0-40bf-b8f1-eb6575e04e22 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:56.302572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:56.316268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:56.329939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd257a-7084-4c09-880f-cb76cbf30fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:56.330759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd257a-7084-4c09-880f-cb76cbf30fd1 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:56.331092 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd257a-7084-4c09-880f-cb76cbf30fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:56.331445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd257a-7084-4c09-880f-cb76cbf30fd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:56.343016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd257a-7084-4c09-880f-cb76cbf30fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-e4cd257a-7084-4c09-880f-cb76cbf30fd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:56.352139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:56.353104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:38:56.353887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 57/109] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:38:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:38:56.359335 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd257a-7084-4c09-880f-cb76cbf30fd1 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:56.474714 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd257a-7084-4c09-880f-cb76cbf30fd1 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:56.519575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef13bc5-c1cb-425a-800d-ecb384836f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:56.520266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef13bc5-c1cb-425a-800d-ecb384836f93 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:56.520609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef13bc5-c1cb-425a-800d-ecb384836f93 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:56.520894 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef13bc5-c1cb-425a-800d-ecb384836f93 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:56.521656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef13bc5-c1cb-425a-800d-ecb384836f93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:56.522505 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef13bc5-c1cb-425a-800d-ecb384836f93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:56.524303 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef13bc5-c1cb-425a-800d-ecb384836f93 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:56.533040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff9d66-ebe2-4e4f-b261-a97a5e2fc56f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:56.535503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 58/110] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:38:56 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:38:56.545320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef13bc5-c1cb-425a-800d-ecb384836f93 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:56.606407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:56.659562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c011cf-d9dd-4cd0-adae-6d6902494e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:56.660395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c011cf-d9dd-4cd0-adae-6d6902494e5b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:56.660683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c011cf-d9dd-4cd0-adae-6d6902494e5b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:56.660957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c011cf-d9dd-4cd0-adae-6d6902494e5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:56.665122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef13bc5-c1cb-425a-800d-ecb384836f93 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:56.680366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c011cf-d9dd-4cd0-adae-6d6902494e5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-53c011cf-d9dd-4cd0-adae-6d6902494e5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:56.696927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:56.698251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:38:56.699045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 53/111] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:38:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 467 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:38:56.704228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598658f-46c9-45b9-98f6-886667fd2c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:56.705501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598658f-46c9-45b9-98f6-886667fd2c76 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:56.705782 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598658f-46c9-45b9-98f6-886667fd2c76 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:56.706056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598658f-46c9-45b9-98f6-886667fd2c76 None admin] PATH_INFO: `/v3/users/9845c1dd09f34a0aa117c5e212e20304` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:56.707876 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c011cf-d9dd-4cd0-adae-6d6902494e5b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:56.711627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598658f-46c9-45b9-98f6-886667fd2c76 None admin] RBAC: Authorizing `identity:get_user(user_id=9845c1dd09f34a0aa117c5e212e20304)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:56.713287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598658f-46c9-45b9-98f6-886667fd2c76 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:56.715470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 59/112] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:56 2022] GET /identity/v3/users/9845c1dd09f34a0aa117c5e212e20304 => generated 248 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:38:56.732530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598658f-46c9-45b9-98f6-886667fd2c76 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:56.877792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:56.904832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598658f-46c9-45b9-98f6-886667fd2c76 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:56.910292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c011cf-d9dd-4cd0-adae-6d6902494e5b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:56.920767 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:56.955187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6fe54-d854-4ff2-82f0-514e683988e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:56.955877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6fe54-d854-4ff2-82f0-514e683988e7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:56.956160 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6fe54-d854-4ff2-82f0-514e683988e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:56.956437 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6fe54-d854-4ff2-82f0-514e683988e7 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:56.957193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6fe54-d854-4ff2-82f0-514e683988e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:56.957498 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6fe54-d854-4ff2-82f0-514e683988e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:56.958926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6fe54-d854-4ff2-82f0-514e683988e7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:56.978777 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 54/113] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:38:56 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:56.980037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5189e2-a4ad-4fea-bc75-972c3ccac43e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:56.980294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5189e2-a4ad-4fea-bc75-972c3ccac43e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:56.980572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5189e2-a4ad-4fea-bc75-972c3ccac43e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:56.981949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 55/114] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:56.985151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95101a3b-3060-4ee2-a7ae-016633c3128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:56.985975 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95101a3b-3060-4ee2-a7ae-016633c3128f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:56.986256 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95101a3b-3060-4ee2-a7ae-016633c3128f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:56.986528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95101a3b-3060-4ee2-a7ae-016633c3128f None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:56.987900 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5189e2-a4ad-4fea-bc75-972c3ccac43e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:56.989153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95101a3b-3060-4ee2-a7ae-016633c3128f None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:56.991537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95101a3b-3060-4ee2-a7ae-016633c3128f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:56.996045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 60/115] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:38:56 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:38:56.997185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e9ab2-623a-4611-bdd3-8dc461e41101 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:56.997472 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e9ab2-623a-4611-bdd3-8dc461e41101 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:56.997747 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e9ab2-623a-4611-bdd3-8dc461e41101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:57.032938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-093e9ab2-623a-4611-bdd3-8dc461e41101 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:57.034234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-093e9ab2-623a-4611-bdd3-8dc461e41101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:57.148182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 61/116] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:57.154862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5189e2-a4ad-4fea-bc75-972c3ccac43e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:57.156102 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e9ab2-623a-4611-bdd3-8dc461e41101 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:57.205665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a53acc-ea7c-4980-94e3-36aefec42a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:57.206547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a53acc-ea7c-4980-94e3-36aefec42a5f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:57.206862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a53acc-ea7c-4980-94e3-36aefec42a5f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:57.207126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a53acc-ea7c-4980-94e3-36aefec42a5f None admin] PATH_INFO: `/v3/users/7201ba55061e4bf4882217b64adb67b6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:57.208311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a53acc-ea7c-4980-94e3-36aefec42a5f None admin] RBAC: Authorizing `identity:get_user(user_id=7201ba55061e4bf4882217b64adb67b6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:57.210255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a53acc-ea7c-4980-94e3-36aefec42a5f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:57.212583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 56/117] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:56 2022] GET /identity/v3/users/7201ba55061e4bf4882217b64adb67b6 => generated 276 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:57.214909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e9232-806c-4399-b379-6a448523b302 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:57.214909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e9232-806c-4399-b379-6a448523b302 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:57.214909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e9232-806c-4399-b379-6a448523b302 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:57.216257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 57/118] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:57.218987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e9232-806c-4399-b379-6a448523b302 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:57.233214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:57.281707 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e9ab2-623a-4611-bdd3-8dc461e41101 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:57.291372 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:57.319400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddf7ad-85e2-4508-aca2-dbaf8a3bb815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:57.320513 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddf7ad-85e2-4508-aca2-dbaf8a3bb815 None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:57.320800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddf7ad-85e2-4508-aca2-dbaf8a3bb815 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:57.321073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddf7ad-85e2-4508-aca2-dbaf8a3bb815 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce/users/9845c1dd09f34a0aa117c5e212e20304/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:57.323179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cddf7ad-85e2-4508-aca2-dbaf8a3bb815 None admin] RBAC: Authorizing `identity:create_grant(project_id=8611e940b0fa4f8d910441467b31bdce, user_id=9845c1dd09f34a0aa117c5e212e20304, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:57.324621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cddf7ad-85e2-4508-aca2-dbaf8a3bb815 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:57.336998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e9232-806c-4399-b379-6a448523b302 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:57.371572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13ee01-f2fa-4a98-b1c9-85cdb98891a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:57.372370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13ee01-f2fa-4a98-b1c9-85cdb98891a3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:57.372648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13ee01-f2fa-4a98-b1c9-85cdb98891a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:57.372919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13ee01-f2fa-4a98-b1c9-85cdb98891a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:57.383785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13ee01-f2fa-4a98-b1c9-85cdb98891a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-5c13ee01-f2fa-4a98-b1c9-85cdb98891a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:57.391706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:57.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:38:57.393567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 58/119] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:38:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:38:57.394208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b98db-9416-4fa5-ad5f-8d2816012d60 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:57.394273 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b98db-9416-4fa5-ad5f-8d2816012d60 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:57.394800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b98db-9416-4fa5-ad5f-8d2816012d60 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:57.398857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 59/120] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:57.398857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b98db-9416-4fa5-ad5f-8d2816012d60 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:57.411242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 62/121] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:38:57 2022] PUT /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce/users/9845c1dd09f34a0aa117c5e212e20304/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:38:57.419082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c7078-ee48-4730-8449-a8787e2063e3 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:57.419082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c7078-ee48-4730-8449-a8787e2063e3 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:57.419082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c7078-ee48-4730-8449-a8787e2063e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:57.435583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-372c7078-ee48-4730-8449-a8787e2063e3 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:57.437257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-372c7078-ee48-4730-8449-a8787e2063e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:57.530595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 63/122] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:57.532076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:57.532596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c317e-5eac-4248-9366-4bf859ece855 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:57.533679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c317e-5eac-4248-9366-4bf859ece855 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:57.534157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c317e-5eac-4248-9366-4bf859ece855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:57.552953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c14c317e-5eac-4248-9366-4bf859ece855 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:57.554884 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c14c317e-5eac-4248-9366-4bf859ece855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:57.571621 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b98db-9416-4fa5-ad5f-8d2816012d60 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:57.611552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa268646-1e67-4e7f-9976-2e714e3f03b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:57.612480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa268646-1e67-4e7f-9976-2e714e3f03b2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:57.612764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa268646-1e67-4e7f-9976-2e714e3f03b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:57.613044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa268646-1e67-4e7f-9976-2e714e3f03b2 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:57.614421 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa268646-1e67-4e7f-9976-2e714e3f03b2 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:57.616080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa268646-1e67-4e7f-9976-2e714e3f03b2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:57.618216 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 60/123] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:38:57 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:57.623949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa268646-1e67-4e7f-9976-2e714e3f03b2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:57.649188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 64/124] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:57.650473 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c317e-5eac-4248-9366-4bf859ece855 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:57.688706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:57.760542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa268646-1e67-4e7f-9976-2e714e3f03b2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:57.770539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c317e-5eac-4248-9366-4bf859ece855 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:57.804747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7ed5d-db11-4a27-b9b2-b1ab4fd88615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:57.805356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7ed5d-db11-4a27-b9b2-b1ab4fd88615 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:57.805643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7ed5d-db11-4a27-b9b2-b1ab4fd88615 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:57.806244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7ed5d-db11-4a27-b9b2-b1ab4fd88615 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:57.807299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7ed5d-db11-4a27-b9b2-b1ab4fd88615 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:57.807619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7ed5d-db11-4a27-b9b2-b1ab4fd88615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:57.809161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7ed5d-db11-4a27-b9b2-b1ab4fd88615 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:57.815111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:57.815404 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366dee8-8dd1-4f4f-aa7e-73cafc26ba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:57.817358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366dee8-8dd1-4f4f-aa7e-73cafc26ba08 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:57.817831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366dee8-8dd1-4f4f-aa7e-73cafc26ba08 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:57.818282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366dee8-8dd1-4f4f-aa7e-73cafc26ba08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:57.819835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 61/125] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:38:57 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:57.820753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7ed5d-db11-4a27-b9b2-b1ab4fd88615 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:57.832728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f366dee8-8dd1-4f4f-aa7e-73cafc26ba08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-f366dee8-8dd1-4f4f-aa7e-73cafc26ba08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:57.842219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:57.843200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:57.844176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:57.844176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:57.844176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:57.844176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:57.844176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:57.844176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:57.844176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:57.844176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:57.844176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:57.844176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:57.844176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:38:57.844176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:38:57.846611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 65/126] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:38:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:38:57.847736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366dee8-8dd1-4f4f-aa7e-73cafc26ba08 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:57.944641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7ed5d-db11-4a27-b9b2-b1ab4fd88615 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:57.952580 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:57.970011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366dee8-8dd1-4f4f-aa7e-73cafc26ba08 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:57.988257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb66af-0973-4911-bae4-48121c9cc9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:57.989362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb66af-0973-4911-bae4-48121c9cc9ac None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:57.989719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb66af-0973-4911-bae4-48121c9cc9ac None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:57.990542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb66af-0973-4911-bae4-48121c9cc9ac None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce/users/7201ba55061e4bf4882217b64adb67b6/roles/582f59fc05ce474287c6fdc503f9ac82` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:57.993377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb66af-0973-4911-bae4-48121c9cc9ac None admin] RBAC: Authorizing `identity:create_grant(project_id=8611e940b0fa4f8d910441467b31bdce, user_id=7201ba55061e4bf4882217b64adb67b6, role_id=582f59fc05ce474287c6fdc503f9ac82)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:57.995448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb66af-0973-4911-bae4-48121c9cc9ac None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:58.014586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc149be-71cb-4ae1-b09f-d2fa551e5201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:58.016047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc149be-71cb-4ae1-b09f-d2fa551e5201 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:58.016335 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc149be-71cb-4ae1-b09f-d2fa551e5201 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:58.016610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc149be-71cb-4ae1-b09f-d2fa551e5201 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:58.017910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc149be-71cb-4ae1-b09f-d2fa551e5201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:58.019443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc149be-71cb-4ae1-b09f-d2fa551e5201 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:58.021289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 66/127] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:38:57 2022] GET /identity/v3/domains/default => generated 200 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:58.022363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc149be-71cb-4ae1-b09f-d2fa551e5201 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:58.066395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 62/128] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:38:57 2022] PUT /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce/users/7201ba55061e4bf4882217b64adb67b6/roles/582f59fc05ce474287c6fdc503f9ac82 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:38:58.067632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb66af-0973-4911-bae4-48121c9cc9ac None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:58.164887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:58.222335 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb66af-0973-4911-bae4-48121c9cc9ac None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:58.249762 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc149be-71cb-4ae1-b09f-d2fa551e5201 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:58.275553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952d12a-d89a-4f59-9572-aa1b1e065e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:58.277921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952d12a-d89a-4f59-9572-aa1b1e065e7f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:58.279233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952d12a-d89a-4f59-9572-aa1b1e065e7f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:58.279689 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952d12a-d89a-4f59-9572-aa1b1e065e7f None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:58.281588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952d12a-d89a-4f59-9572-aa1b1e065e7f None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:58.282430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952d12a-d89a-4f59-9572-aa1b1e065e7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:58.285083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952d12a-d89a-4f59-9572-aa1b1e065e7f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:58.310913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 63/129] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:38:58 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:58.311732 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952d12a-d89a-4f59-9572-aa1b1e065e7f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:58.316537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fbf7b-f7cd-4121-b92d-0f9f13135352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:58.317708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fbf7b-f7cd-4121-b92d-0f9f13135352 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:58.318193 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fbf7b-f7cd-4121-b92d-0f9f13135352 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:58.318989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fbf7b-f7cd-4121-b92d-0f9f13135352 None admin] PATH_INFO: `/v3/users/dbda967853b6446480e1a026eab76ae0` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:58.339184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4fbf7b-f7cd-4121-b92d-0f9f13135352 None admin] RBAC: Authorizing `identity:get_user(user_id=dbda967853b6446480e1a026eab76ae0)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:58.342165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4fbf7b-f7cd-4121-b92d-0f9f13135352 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:58.344593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 67/130] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:58 2022] GET /identity/v3/users/dbda967853b6446480e1a026eab76ae0 => generated 290 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:58.345589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fbf7b-f7cd-4121-b92d-0f9f13135352 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:58.365009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:58.460391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952d12a-d89a-4f59-9572-aa1b1e065e7f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:58.486715 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:58.488080 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fbf7b-f7cd-4121-b92d-0f9f13135352 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:58.500732 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec72b3-fb3c-4630-a8d1-ea9aed3a59b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:58.501767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec72b3-fb3c-4630-a8d1-ea9aed3a59b7 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:58.502262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec72b3-fb3c-4630-a8d1-ea9aed3a59b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:58.502805 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec72b3-fb3c-4630-a8d1-ea9aed3a59b7 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:58.503781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec72b3-fb3c-4630-a8d1-ea9aed3a59b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:58.505474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec72b3-fb3c-4630-a8d1-ea9aed3a59b7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:58.542060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2ca1e-1440-4c32-94fa-676c148ee561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:58.543612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2ca1e-1440-4c32-94fa-676c148ee561 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:58.544101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2ca1e-1440-4c32-94fa-676c148ee561 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:58.544638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2ca1e-1440-4c32-94fa-676c148ee561 None admin] PATH_INFO: `/v3/users/7201ba55061e4bf4882217b64adb67b6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:58.546192 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2ca1e-1440-4c32-94fa-676c148ee561 None admin] RBAC: Authorizing `identity:get_user(user_id=7201ba55061e4bf4882217b64adb67b6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:58.547937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2ca1e-1440-4c32-94fa-676c148ee561 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:58.550268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 68/131] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:58 2022] GET /identity/v3/users/7201ba55061e4bf4882217b64adb67b6 => generated 276 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:58.551358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2ca1e-1440-4c32-94fa-676c148ee561 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:58.591610 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 64/132] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:38:58 2022] POST /identity/v3/projects => generated 303 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:38:58.592988 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec72b3-fb3c-4630-a8d1-ea9aed3a59b7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:58.717184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:58.767804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2ca1e-1440-4c32-94fa-676c148ee561 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:58.827730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141c40d-6119-4975-aabe-dc77c68e487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:58.828802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141c40d-6119-4975-aabe-dc77c68e487e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:58.829309 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141c40d-6119-4975-aabe-dc77c68e487e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:58.829788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141c40d-6119-4975-aabe-dc77c68e487e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:58.833980 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec72b3-fb3c-4630-a8d1-ea9aed3a59b7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:58.916203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10330c1d-244a-4d8b-87fb-1b460db8308a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:58.917748 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10330c1d-244a-4d8b-87fb-1b460db8308a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:58.919258 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10330c1d-244a-4d8b-87fb-1b460db8308a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:58.920399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10330c1d-244a-4d8b-87fb-1b460db8308a None admin] PATH_INFO: `/v3/projects/5480346b7e284479ab3ab16d176d570f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:58.922238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10330c1d-244a-4d8b-87fb-1b460db8308a None admin] RBAC: Authorizing `identity:get_project(project_id=5480346b7e284479ab3ab16d176d570f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:58.924147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10330c1d-244a-4d8b-87fb-1b460db8308a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:58.927661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 65/133] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:38:58 2022] GET /identity/v3/projects/5480346b7e284479ab3ab16d176d570f => generated 313 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:58.934239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10330c1d-244a-4d8b-87fb-1b460db8308a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:58.986996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:58.987883 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141c40d-6119-4975-aabe-dc77c68e487e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:58.989378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141c40d-6119-4975-aabe-dc77c68e487e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:59.050597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10330c1d-244a-4d8b-87fb-1b460db8308a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:59.089035 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4a99a-1f57-4a45-a676-ae567d11dc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:59.090346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4a99a-1f57-4a45-a676-ae567d11dc97 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:59.090865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4a99a-1f57-4a45-a676-ae567d11dc97 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:59.091826 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4a99a-1f57-4a45-a676-ae567d11dc97 None admin] PATH_INFO: `/v3/projects/5480346b7e284479ab3ab16d176d570f/users/7201ba55061e4bf4882217b64adb67b6/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:59.094079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4a99a-1f57-4a45-a676-ae567d11dc97 None admin] RBAC: Authorizing `identity:create_grant(project_id=5480346b7e284479ab3ab16d176d570f, user_id=7201ba55061e4bf4882217b64adb67b6, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:59.096202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4a99a-1f57-4a45-a676-ae567d11dc97 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:59.145686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:59.170956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 66/134] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:38:58 2022] PUT /identity/v3/projects/5480346b7e284479ab3ab16d176d570f/users/7201ba55061e4bf4882217b64adb67b6/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:38:59.172358 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2db1eb-37f8-426f-89a7-0049de63023c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:59.172705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2db1eb-37f8-426f-89a7-0049de63023c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:59.172992 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2db1eb-37f8-426f-89a7-0049de63023c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:59.174309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 67/135] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:38:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:38:59.177288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 69/136] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:38:58 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:38:59.180930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fe4dd-0873-4f63-954e-6031e1642d05 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:59.181531 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fe4dd-0873-4f63-954e-6031e1642d05 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:59.182038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fe4dd-0873-4f63-954e-6031e1642d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:59.186865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141c40d-6119-4975-aabe-dc77c68e487e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:59.224080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f81fe4dd-0873-4f63-954e-6031e1642d05 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:38:59.226533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f81fe4dd-0873-4f63-954e-6031e1642d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:38:59.300602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141c40d-6119-4975-aabe-dc77c68e487e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:59.339228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1cc81-f9f8-48b3-94e9-f413784509a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:59.340483 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1cc81-f9f8-48b3-94e9-f413784509a7 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:59.340864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1cc81-f9f8-48b3-94e9-f413784509a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:59.341244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1cc81-f9f8-48b3-94e9-f413784509a7 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:59.342559 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1cc81-f9f8-48b3-94e9-f413784509a7 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:59.344182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1cc81-f9f8-48b3-94e9-f413784509a7 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:59.346009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 70/137] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:38:59 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:38:59.352930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1cc81-f9f8-48b3-94e9-f413784509a7 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:59.364956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 68/138] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:38:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:38:59.404449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fe4dd-0873-4f63-954e-6031e1642d05 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:59.411735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:59.466741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1cc81-f9f8-48b3-94e9-f413784509a7 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:59.488022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:59.503812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f32ba9-ce5c-4223-9d9e-1ad4d19aeb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:59.504512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f32ba9-ce5c-4223-9d9e-1ad4d19aeb00 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:59.504801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f32ba9-ce5c-4223-9d9e-1ad4d19aeb00 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:59.505081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f32ba9-ce5c-4223-9d9e-1ad4d19aeb00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:59.506102 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f32ba9-ce5c-4223-9d9e-1ad4d19aeb00 None admin] RBAC: Adding query filter params (role.id=6d1a0644f0bf45b4aa0f9fd0b3843261, scope.project.id=8611e940b0fa4f8d910441467b31bdce, user.id=dbda967853b6446480e1a026eab76ae0) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:59.506464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f32ba9-ce5c-4223-9d9e-1ad4d19aeb00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:59.508186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f32ba9-ce5c-4223-9d9e-1ad4d19aeb00 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:59.518188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 71/139] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:38:59 2022] GET /identity/v3/role_assignments?user.id=dbda967853b6446480e1a026eab76ae0&scope.project.id=8611e940b0fa4f8d910441467b31bdce&role.id=6d1a0644f0bf45b4aa0f9fd0b3843261 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:59.622457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fe4dd-0873-4f63-954e-6031e1642d05 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:59.670930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76fa37-d564-4e2b-a1ed-ea2c40a2ad3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:59.672157 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76fa37-d564-4e2b-a1ed-ea2c40a2ad3f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:59.672640 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76fa37-d564-4e2b-a1ed-ea2c40a2ad3f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:59.673133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76fa37-d564-4e2b-a1ed-ea2c40a2ad3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:59.690684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76fa37-d564-4e2b-a1ed-ea2c40a2ad3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-db76fa37-d564-4e2b-a1ed-ea2c40a2ad3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:38:59.699661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:38:59.700573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:38:59.701320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:38:59.701320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:38:59.701320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:38:59.701320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:38:59.701320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:38:59.701320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:38:59.701320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:38:59.701320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:38:59.701320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:38:59.701320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:38:59.701320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:38:59.701320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:38:59.707934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 69/140] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:38:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:38:59.714863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f32ba9-ce5c-4223-9d9e-1ad4d19aeb00 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:38:59.841652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f32ba9-ce5c-4223-9d9e-1ad4d19aeb00 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:38:59.856660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:38:59.886466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5c8be-7302-4cb6-9420-f59ed1767161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:38:59.887182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5c8be-7302-4cb6-9420-f59ed1767161 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:38:59.887476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5c8be-7302-4cb6-9420-f59ed1767161 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:38:59.887758 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5c8be-7302-4cb6-9420-f59ed1767161 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:38:59.888516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5c8be-7302-4cb6-9420-f59ed1767161 None admin] RBAC: Adding query filter params (name=member) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:38:59.888827 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5c8be-7302-4cb6-9420-f59ed1767161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:38:59.891130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5c8be-7302-4cb6-9420-f59ed1767161 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:38:59.905805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 72/141] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:38:59 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:38:59.911541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76fa37-d564-4e2b-a1ed-ea2c40a2ad3f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:00.027106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:00.040263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76fa37-d564-4e2b-a1ed-ea2c40a2ad3f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:00.076136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f3323-69df-4c7b-abd1-c7b2f41da1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:00.077020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f3323-69df-4c7b-abd1-c7b2f41da1e8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:00.077300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f3323-69df-4c7b-abd1-c7b2f41da1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:00.077575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f3323-69df-4c7b-abd1-c7b2f41da1e8 None admin] PATH_INFO: `/v3/users/7201ba55061e4bf4882217b64adb67b6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:00.078775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f3323-69df-4c7b-abd1-c7b2f41da1e8 None admin] RBAC: Authorizing `identity:get_user(user_id=7201ba55061e4bf4882217b64adb67b6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:00.080244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f3323-69df-4c7b-abd1-c7b2f41da1e8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:00.081856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 70/142] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:38:59 2022] GET /identity/v3/users/7201ba55061e4bf4882217b64adb67b6 => generated 276 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:00.088092 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5c8be-7302-4cb6-9420-f59ed1767161 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:00.171469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54544a85-60fc-4951-9c04-5ee27f35221c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:00.171796 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54544a85-60fc-4951-9c04-5ee27f35221c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:00.172032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54544a85-60fc-4951-9c04-5ee27f35221c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:00.173534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 71/143] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:00.184665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3818b-219c-46ef-8e89-679aaef01b3f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:00.184953 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3818b-219c-46ef-8e89-679aaef01b3f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:00.185222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3818b-219c-46ef-8e89-679aaef01b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:00.223213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-27d3818b-219c-46ef-8e89-679aaef01b3f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:00.225442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-27d3818b-219c-46ef-8e89-679aaef01b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:00.257814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5c8be-7302-4cb6-9420-f59ed1767161 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:00.300260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca3a4d-54fa-4bc6-8e40-883ffa8ead2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:00.300922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca3a4d-54fa-4bc6-8e40-883ffa8ead2d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:00.301203 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca3a4d-54fa-4bc6-8e40-883ffa8ead2d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:00.301480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca3a4d-54fa-4bc6-8e40-883ffa8ead2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:00.333677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 72/144] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:00.362566 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3818b-219c-46ef-8e89-679aaef01b3f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:00.372562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:00.415877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca3a4d-54fa-4bc6-8e40-883ffa8ead2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:00.417713 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca3a4d-54fa-4bc6-8e40-883ffa8ead2d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:00.469058 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:00.479167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3818b-219c-46ef-8e89-679aaef01b3f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:00.516380 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276625dd-7e2c-42d5-94ab-42c56d87d788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:00.517281 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276625dd-7e2c-42d5-94ab-42c56d87d788 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:00.517641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276625dd-7e2c-42d5-94ab-42c56d87d788 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:00.517990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276625dd-7e2c-42d5-94ab-42c56d87d788 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:00.538094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276625dd-7e2c-42d5-94ab-42c56d87d788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-276625dd-7e2c-42d5-94ab-42c56d87d788 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:00.546389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:00.547277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:00.548115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:00.548115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:00.548115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:00.548115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:00.548115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:00.548115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:00.548115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:00.548115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:00.548115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:00.548115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:00.548115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:00.548115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:00.554468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 73/145] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:39:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:00.562692 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276625dd-7e2c-42d5-94ab-42c56d87d788 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:00.620338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 73/146] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:00 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:00.624142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca3a4d-54fa-4bc6-8e40-883ffa8ead2d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:00.745390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276625dd-7e2c-42d5-94ab-42c56d87d788 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:00.768312 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:00.782559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28f339-ba24-434b-b527-eaabc31ed88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:00.783376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28f339-ba24-434b-b527-eaabc31ed88f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:00.783665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28f339-ba24-434b-b527-eaabc31ed88f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:00.783979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28f339-ba24-434b-b527-eaabc31ed88f None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:00.784783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28f339-ba24-434b-b527-eaabc31ed88f None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:00.785092 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28f339-ba24-434b-b527-eaabc31ed88f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:00.786498 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28f339-ba24-434b-b527-eaabc31ed88f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:00.795957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 74/147] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:00 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:00.796197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca3a4d-54fa-4bc6-8e40-883ffa8ead2d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:00.801710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28f339-ba24-434b-b527-eaabc31ed88f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:00.823192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:00.840888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c55ec0-3e65-41e5-a177-0b8dcdc03892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:00.841852 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c55ec0-3e65-41e5-a177-0b8dcdc03892 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:00.842136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c55ec0-3e65-41e5-a177-0b8dcdc03892 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:00.842439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c55ec0-3e65-41e5-a177-0b8dcdc03892 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:00.844453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c55ec0-3e65-41e5-a177-0b8dcdc03892 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:00.845968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c55ec0-3e65-41e5-a177-0b8dcdc03892 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:00.848051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 74/148] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:00 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:00.855075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c55ec0-3e65-41e5-a177-0b8dcdc03892 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:00.927254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28f339-ba24-434b-b527-eaabc31ed88f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:00.990311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96b6f4-61d6-46c6-ae43-dc368f2a9d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:00.991927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96b6f4-61d6-46c6-ae43-dc368f2a9d7d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:00.992222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96b6f4-61d6-46c6-ae43-dc368f2a9d7d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:00.992553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96b6f4-61d6-46c6-ae43-dc368f2a9d7d None admin] PATH_INFO: `/v3/users/9845c1dd09f34a0aa117c5e212e20304` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:00.999455 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96b6f4-61d6-46c6-ae43-dc368f2a9d7d None admin] RBAC: Authorizing `identity:get_user(user_id=9845c1dd09f34a0aa117c5e212e20304)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:01.000978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96b6f4-61d6-46c6-ae43-dc368f2a9d7d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:01.003182 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 75/149] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:00 2022] GET /identity/v3/users/9845c1dd09f34a0aa117c5e212e20304 => generated 248 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:01.009111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96b6f4-61d6-46c6-ae43-dc368f2a9d7d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:01.033841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c55ec0-3e65-41e5-a177-0b8dcdc03892 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:01.083611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05107e6c-e504-49df-8fef-cca63faf1518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:01.084953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05107e6c-e504-49df-8fef-cca63faf1518 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:01.085387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05107e6c-e504-49df-8fef-cca63faf1518 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:01.085668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05107e6c-e504-49df-8fef-cca63faf1518 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:01.086516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05107e6c-e504-49df-8fef-cca63faf1518 None admin] RBAC: Adding query filter params (role.id=e87c07d8d70d46d89a2c4f7246edeff6, scope.project.id=8611e940b0fa4f8d910441467b31bdce, user.id=7201ba55061e4bf4882217b64adb67b6) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:01.086963 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05107e6c-e504-49df-8fef-cca63faf1518 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:01.088529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05107e6c-e504-49df-8fef-cca63faf1518 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:01.102855 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 75/150] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:00 2022] GET /identity/v3/role_assignments?user.id=7201ba55061e4bf4882217b64adb67b6&scope.project.id=8611e940b0fa4f8d910441467b31bdce&role.id=e87c07d8d70d46d89a2c4f7246edeff6 => generated 610 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:01.155197 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:01.176066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96b6f4-61d6-46c6-ae43-dc368f2a9d7d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:01.219791 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b16cc-ae44-4e89-84a5-2d544fcf199e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:01.220398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b16cc-ae44-4e89-84a5-2d544fcf199e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:01.220688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b16cc-ae44-4e89-84a5-2d544fcf199e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:01.220973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b16cc-ae44-4e89-84a5-2d544fcf199e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:01.313541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b16cc-ae44-4e89-84a5-2d544fcf199e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:01.315081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b16cc-ae44-4e89-84a5-2d544fcf199e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:01.408993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:01.434941 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 76/151] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:01 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:01.439966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05107e6c-e504-49df-8fef-cca63faf1518 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:01.461151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:01.577391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05107e6c-e504-49df-8fef-cca63faf1518 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:01.625637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc9614-c5df-4142-b1fc-fe2e20717309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:01.627071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc9614-c5df-4142-b1fc-fe2e20717309 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:01.627567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc9614-c5df-4142-b1fc-fe2e20717309 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:01.628037 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc9614-c5df-4142-b1fc-fe2e20717309 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:01.629450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc9614-c5df-4142-b1fc-fe2e20717309 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:01.631233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc9614-c5df-4142-b1fc-fe2e20717309 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:01.633165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 76/152] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:01 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:01.639543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b16cc-ae44-4e89-84a5-2d544fcf199e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:01.781149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b16cc-ae44-4e89-84a5-2d544fcf199e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:01.823646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc5165-3e50-44bd-991a-9961cf2de262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:01.824319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc5165-3e50-44bd-991a-9961cf2de262 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:01.824600 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc5165-3e50-44bd-991a-9961cf2de262 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:01.824877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc5165-3e50-44bd-991a-9961cf2de262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:01.825718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc5165-3e50-44bd-991a-9961cf2de262 None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=8611e940b0fa4f8d910441467b31bdce, user.id=9845c1dd09f34a0aa117c5e212e20304) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:01.826030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc5165-3e50-44bd-991a-9961cf2de262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:01.827591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc5165-3e50-44bd-991a-9961cf2de262 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:01.840127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758df47-b22b-402b-bb69-ad8f8dca05fa None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:01.840761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758df47-b22b-402b-bb69-ad8f8dca05fa None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:01.841323 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758df47-b22b-402b-bb69-ad8f8dca05fa None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:01.842909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 77/153] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:01 2022] GET /identity/v3/role_assignments?user.id=9845c1dd09f34a0aa117c5e212e20304&scope.project.id=8611e940b0fa4f8d910441467b31bdce&role.id=4f574961798246a4b12ad2276e67685d => generated 610 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:01.843248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 77/154] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:01 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:01.851722 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f0dda-6b35-4098-80d1-a9685fec540b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:01.852015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f0dda-6b35-4098-80d1-a9685fec540b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:01.852291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f0dda-6b35-4098-80d1-a9685fec540b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:01.880063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ae5f0dda-6b35-4098-80d1-a9685fec540b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:01.882217 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ae5f0dda-6b35-4098-80d1-a9685fec540b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:01.934358 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:01.979173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 78/155] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:02.023741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758df47-b22b-402b-bb69-ad8f8dca05fa None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:02.198334 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:02.284685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758df47-b22b-402b-bb69-ad8f8dca05fa None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:02.353399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30f6db-5f87-4af9-9cdb-54d29a7a47c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:02.354226 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30f6db-5f87-4af9-9cdb-54d29a7a47c9 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:02.354511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30f6db-5f87-4af9-9cdb-54d29a7a47c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:02.354788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30f6db-5f87-4af9-9cdb-54d29a7a47c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:02.366443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30f6db-5f87-4af9-9cdb-54d29a7a47c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-8f30f6db-5f87-4af9-9cdb-54d29a7a47c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:02.374860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:02.375735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:02.376507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:02.376507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:02.376507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:02.376507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:02.376507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:02.376507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:02.376507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:02.376507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:02.376507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:02.376507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:02.376507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:02.376507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:02.379100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 78/156] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:39:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:02.393976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f0dda-6b35-4098-80d1-a9685fec540b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:02.502453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e4062-4ccc-46a6-9ffa-e3111c5649ab None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:02.503007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e4062-4ccc-46a6-9ffa-e3111c5649ab None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:02.503007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e4062-4ccc-46a6-9ffa-e3111c5649ab None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:02.504427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 79/157] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:39:02.512627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303e741-7dc2-4388-a56b-619723a041bc None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:02.512627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303e741-7dc2-4388-a56b-619723a041bc None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:02.512627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303e741-7dc2-4388-a56b-619723a041bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:02.562679 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f0dda-6b35-4098-80d1-a9685fec540b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:02.568194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c303e741-7dc2-4388-a56b-619723a041bc None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:02.570021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c303e741-7dc2-4388-a56b-619723a041bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:02.608360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddc056-5ecc-4751-9a05-8ef8f73b6d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:02.609475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddc056-5ecc-4751-9a05-8ef8f73b6d61 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:02.609761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddc056-5ecc-4751-9a05-8ef8f73b6d61 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:02.610035 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddc056-5ecc-4751-9a05-8ef8f73b6d61 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:02.611096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddc056-5ecc-4751-9a05-8ef8f73b6d61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:02.611468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddc056-5ecc-4751-9a05-8ef8f73b6d61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:02.613271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddc056-5ecc-4751-9a05-8ef8f73b6d61 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:02.625651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 79/158] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:02 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:02.631183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddc056-5ecc-4751-9a05-8ef8f73b6d61 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:02.658734 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 80/159] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:02.679053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:02.685109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303e741-7dc2-4388-a56b-619723a041bc None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:02.759436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddc056-5ecc-4751-9a05-8ef8f73b6d61 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:02.765317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:02.804666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98efe3b-f876-41ce-9cd8-6a69e8c6b09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:02.805482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98efe3b-f876-41ce-9cd8-6a69e8c6b09d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:02.805982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98efe3b-f876-41ce-9cd8-6a69e8c6b09d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:02.806087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98efe3b-f876-41ce-9cd8-6a69e8c6b09d None admin] PATH_INFO: `/v3/users/7201ba55061e4bf4882217b64adb67b6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:02.807270 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98efe3b-f876-41ce-9cd8-6a69e8c6b09d None admin] RBAC: Authorizing `identity:get_user(user_id=7201ba55061e4bf4882217b64adb67b6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:02.808736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98efe3b-f876-41ce-9cd8-6a69e8c6b09d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:02.814905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 80/160] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:02 2022] GET /identity/v3/users/7201ba55061e4bf4882217b64adb67b6 => generated 276 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:02.814905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa28d66-d046-49c9-9c7d-f6f9a26792ba None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:02.814905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa28d66-d046-49c9-9c7d-f6f9a26792ba None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:02.814905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa28d66-d046-49c9-9c7d-f6f9a26792ba None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:02.814905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 81/161] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:02 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:02.818475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa28d66-d046-49c9-9c7d-f6f9a26792ba None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:02.819932 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303e741-7dc2-4388-a56b-619723a041bc None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:02.858178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e3a20-5ede-4c51-ad18-a07a47330c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:02.859026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e3a20-5ede-4c51-ad18-a07a47330c6b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:02.859304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e3a20-5ede-4c51-ad18-a07a47330c6b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:02.859578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e3a20-5ede-4c51-ad18-a07a47330c6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:02.860758 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e3a20-5ede-4c51-ad18-a07a47330c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:02.862214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e3a20-5ede-4c51-ad18-a07a47330c6b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:02.864015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 81/162] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:02 2022] GET /identity/v3/domains/default => generated 200 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:02.865302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053b39c-e6f1-4e71-a5c2-327d05a82c2e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:02.865553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053b39c-e6f1-4e71-a5c2-327d05a82c2e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:02.865831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053b39c-e6f1-4e71-a5c2-327d05a82c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:02.885680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a053b39c-e6f1-4e71-a5c2-327d05a82c2e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:02.886938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a053b39c-e6f1-4e71-a5c2-327d05a82c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:02.953039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa28d66-d046-49c9-9c7d-f6f9a26792ba None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:02.955419 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:02.958157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 82/163] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:02.959643 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053b39c-e6f1-4e71-a5c2-327d05a82c2e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:02.991824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18a20b-7c44-45ec-8b66-78557047002e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:02.992533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18a20b-7c44-45ec-8b66-78557047002e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:02.992817 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18a20b-7c44-45ec-8b66-78557047002e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:02.993085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18a20b-7c44-45ec-8b66-78557047002e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:03.026772 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:03.062242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18a20b-7c44-45ec-8b66-78557047002e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:03.063744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18a20b-7c44-45ec-8b66-78557047002e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:03.064973 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053b39c-e6f1-4e71-a5c2-327d05a82c2e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:03.105418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1851fd-9b8f-4837-9cd0-7c98d1fdebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:03.106224 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1851fd-9b8f-4837-9cd0-7c98d1fdebb7 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:03.106500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1851fd-9b8f-4837-9cd0-7c98d1fdebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:03.106772 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1851fd-9b8f-4837-9cd0-7c98d1fdebb7 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:03.107565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1851fd-9b8f-4837-9cd0-7c98d1fdebb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:03.109051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1851fd-9b8f-4837-9cd0-7c98d1fdebb7 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:03.165085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 82/164] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:02 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:03.166327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeca4cf-01cd-443e-abb6-618d3ca389a0 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:03.166609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeca4cf-01cd-443e-abb6-618d3ca389a0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:03.166968 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeca4cf-01cd-443e-abb6-618d3ca389a0 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:03.168491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 83/165] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:03.172553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeca4cf-01cd-443e-abb6-618d3ca389a0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:03.176870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:03.196917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 83/166] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:39:02 2022] POST /identity/v3/users => generated 284 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:39:03.197776 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1851fd-9b8f-4837-9cd0-7c98d1fdebb7 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:03.301521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeca4cf-01cd-443e-abb6-618d3ca389a0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:03.326978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1851fd-9b8f-4837-9cd0-7c98d1fdebb7 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:03.348026 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c170b44-8b9c-43d2-bb29-67220badbcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:03.348421 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:03.348969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c170b44-8b9c-43d2-bb29-67220badbcae None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:03.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c170b44-8b9c-43d2-bb29-67220badbcae None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:03.349554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c170b44-8b9c-43d2-bb29-67220badbcae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:03.362271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c170b44-8b9c-43d2-bb29-67220badbcae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:03.368570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8047740-748d-4f3f-af42-766b0005fc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:03.369802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8047740-748d-4f3f-af42-766b0005fc3d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:03.370280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8047740-748d-4f3f-af42-766b0005fc3d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-4c170b44-8b9c-43d2-bb29-67220badbcae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:03.370379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:03.371434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:03.377392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:03.377392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:03.377392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:03.377392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:03.377392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:03.377392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:03.377392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:03.377392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:03.377392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:03.377392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:03.377392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:03.377392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:03.377981 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8047740-748d-4f3f-af42-766b0005fc3d None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:03.378052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 84/167] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:03.378124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8047740-748d-4f3f-af42-766b0005fc3d None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:03.378239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845e001-4a43-4aeb-93e7-2f4e12348012 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:03.378239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845e001-4a43-4aeb-93e7-2f4e12348012 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:03.378239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845e001-4a43-4aeb-93e7-2f4e12348012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:03.378427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8047740-748d-4f3f-af42-766b0005fc3d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:03.378427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 84/168] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:03 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:03.378427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8047740-748d-4f3f-af42-766b0005fc3d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:03.395332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b845e001-4a43-4aeb-93e7-2f4e12348012 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:03.397257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b845e001-4a43-4aeb-93e7-2f4e12348012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:03.488331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:03.499146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 85/169] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:03.499906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845e001-4a43-4aeb-93e7-2f4e12348012 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:03.513456 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8047740-748d-4f3f-af42-766b0005fc3d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:03.553092 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf007738-af49-4a26-b5ab-4c4061b8e4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:03.553912 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf007738-af49-4a26-b5ab-4c4061b8e4d9 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:03.554198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf007738-af49-4a26-b5ab-4c4061b8e4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:03.554471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf007738-af49-4a26-b5ab-4c4061b8e4d9 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:03.556204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf007738-af49-4a26-b5ab-4c4061b8e4d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:03.556204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf007738-af49-4a26-b5ab-4c4061b8e4d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:03.557063 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf007738-af49-4a26-b5ab-4c4061b8e4d9 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:03.566441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 85/170] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:03 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:03.567167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf007738-af49-4a26-b5ab-4c4061b8e4d9 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:03.637514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845e001-4a43-4aeb-93e7-2f4e12348012 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:03.645076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:03.685884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a801d2-c920-40b3-a35a-7b192c1cb3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:03.686679 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a801d2-c920-40b3-a35a-7b192c1cb3fc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:03.687885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a801d2-c920-40b3-a35a-7b192c1cb3fc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:03.687885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a801d2-c920-40b3-a35a-7b192c1cb3fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:03.688071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a801d2-c920-40b3-a35a-7b192c1cb3fc None admin] RBAC: Adding query filter params (role.id=582f59fc05ce474287c6fdc503f9ac82, scope.project.id=8611e940b0fa4f8d910441467b31bdce, user.id=7201ba55061e4bf4882217b64adb67b6) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:03.688389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a801d2-c920-40b3-a35a-7b192c1cb3fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:03.689869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a801d2-c920-40b3-a35a-7b192c1cb3fc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:03.698752 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 86/171] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:03 2022] GET /identity/v3/role_assignments?user.id=7201ba55061e4bf4882217b64adb67b6&scope.project.id=8611e940b0fa4f8d910441467b31bdce&role.id=582f59fc05ce474287c6fdc503f9ac82 => generated 610 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:03.699932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a801d2-c920-40b3-a35a-7b192c1cb3fc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:03.723678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf007738-af49-4a26-b5ab-4c4061b8e4d9 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:03.781549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32414ac8-3055-4ef7-a7e9-d2b47684097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:03.782286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32414ac8-3055-4ef7-a7e9-d2b47684097b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:03.782568 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32414ac8-3055-4ef7-a7e9-d2b47684097b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:03.782873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32414ac8-3055-4ef7-a7e9-d2b47684097b None admin] PATH_INFO: `/v3/roles/member` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:03.791917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:03.797170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32414ac8-3055-4ef7-a7e9-d2b47684097b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-32414ac8-3055-4ef7-a7e9-d2b47684097b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:03.805554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:03.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:03.808396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 86/172] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:03.816296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32414ac8-3055-4ef7-a7e9-d2b47684097b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:03.851837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a801d2-c920-40b3-a35a-7b192c1cb3fc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:03.893347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15210547-678a-4a0c-9c00-c80e41ebe5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:03.894609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15210547-678a-4a0c-9c00-c80e41ebe5c6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:03.894741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15210547-678a-4a0c-9c00-c80e41ebe5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:03.896019 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15210547-678a-4a0c-9c00-c80e41ebe5c6 None admin] PATH_INFO: `/v3/users/dbda967853b6446480e1a026eab76ae0` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:03.898374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15210547-678a-4a0c-9c00-c80e41ebe5c6 None admin] RBAC: Authorizing `identity:get_user(user_id=dbda967853b6446480e1a026eab76ae0)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:03.899916 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15210547-678a-4a0c-9c00-c80e41ebe5c6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:03.901960 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 87/173] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:03 2022] GET /identity/v3/users/dbda967853b6446480e1a026eab76ae0 => generated 290 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:03.908032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15210547-678a-4a0c-9c00-c80e41ebe5c6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:03.947280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32414ac8-3055-4ef7-a7e9-d2b47684097b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:03.959697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:03.989231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f85d4-89e7-4bf4-aca7-7fdc5e767f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:03.990945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f85d4-89e7-4bf4-aca7-7fdc5e767f2c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:03.991578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f85d4-89e7-4bf4-aca7-7fdc5e767f2c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:03.992296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f85d4-89e7-4bf4-aca7-7fdc5e767f2c None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:03.993387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f85d4-89e7-4bf4-aca7-7fdc5e767f2c None admin] RBAC: Adding query filter params (name=member) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:03.993941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f85d4-89e7-4bf4-aca7-7fdc5e767f2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:03.995686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f85d4-89e7-4bf4-aca7-7fdc5e767f2c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:04.007179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 87/174] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:03 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:04.012887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f85d4-89e7-4bf4-aca7-7fdc5e767f2c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:04.039219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15210547-678a-4a0c-9c00-c80e41ebe5c6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:04.079081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:04.107059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9f6f9-d8e2-45ca-a0c9-dd3ac91c0a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:04.114842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9f6f9-d8e2-45ca-a0c9-dd3ac91c0a29 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:04.114842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9f6f9-d8e2-45ca-a0c9-dd3ac91c0a29 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:04.114842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9f6f9-d8e2-45ca-a0c9-dd3ac91c0a29 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:04.114842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9f6f9-d8e2-45ca-a0c9-dd3ac91c0a29 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:04.114842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9f6f9-d8e2-45ca-a0c9-dd3ac91c0a29 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:04.117511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 88/175] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:03 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:04.123300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9f6f9-d8e2-45ca-a0c9-dd3ac91c0a29 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:04.209728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f85d4-89e7-4bf4-aca7-7fdc5e767f2c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:04.281486 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c6bb1-2421-46cc-8597-a3f951c09a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:04.282389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c6bb1-2421-46cc-8597-a3f951c09a3e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:04.282667 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c6bb1-2421-46cc-8597-a3f951c09a3e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:04.282968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c6bb1-2421-46cc-8597-a3f951c09a3e None admin] PATH_INFO: `/v3/users/7201ba55061e4bf4882217b64adb67b6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:04.284244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c6bb1-2421-46cc-8597-a3f951c09a3e None admin] RBAC: Authorizing `identity:get_user(user_id=7201ba55061e4bf4882217b64adb67b6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:04.285731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c6bb1-2421-46cc-8597-a3f951c09a3e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:04.288079 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 88/176] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:04 2022] GET /identity/v3/users/7201ba55061e4bf4882217b64adb67b6 => generated 276 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:04.294448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c6bb1-2421-46cc-8597-a3f951c09a3e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:04.332640 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9f6f9-d8e2-45ca-a0c9-dd3ac91c0a29 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:04.372075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:04.405326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c479c2-c739-4783-9b38-9b54bb40e8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:04.406232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c479c2-c739-4783-9b38-9b54bb40e8a2 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:04.406533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c479c2-c739-4783-9b38-9b54bb40e8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:04.406808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c479c2-c739-4783-9b38-9b54bb40e8a2 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce/users/dbda967853b6446480e1a026eab76ae0/roles/6d1a0644f0bf45b4aa0f9fd0b3843261` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:04.410335 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c479c2-c739-4783-9b38-9b54bb40e8a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8611e940b0fa4f8d910441467b31bdce, user_id=dbda967853b6446480e1a026eab76ae0, role_id=6d1a0644f0bf45b4aa0f9fd0b3843261)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:04.411877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c479c2-c739-4783-9b38-9b54bb40e8a2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:04.470735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c6bb1-2421-46cc-8597-a3f951c09a3e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:04.504024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 89/177] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:39:04 2022] PUT /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce/users/dbda967853b6446480e1a026eab76ae0/roles/6d1a0644f0bf45b4aa0f9fd0b3843261 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:39:04.519506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb3d77-3dab-40cf-a2f7-69e0a06f1bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:04.520179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb3d77-3dab-40cf-a2f7-69e0a06f1bad None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:04.520461 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb3d77-3dab-40cf-a2f7-69e0a06f1bad None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:04.520740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb3d77-3dab-40cf-a2f7-69e0a06f1bad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:04.614731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb3d77-3dab-40cf-a2f7-69e0a06f1bad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:04.616126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb3d77-3dab-40cf-a2f7-69e0a06f1bad None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:04.675434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:04.769906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 89/178] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:04 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:04.775791 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c479c2-c739-4783-9b38-9b54bb40e8a2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:04.856711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:04.915302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c479c2-c739-4783-9b38-9b54bb40e8a2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:04.965007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25621eba-9951-496f-bdae-4089adaa7689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:04.965906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25621eba-9951-496f-bdae-4089adaa7689 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:04.966187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25621eba-9951-496f-bdae-4089adaa7689 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:04.966462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25621eba-9951-496f-bdae-4089adaa7689 None admin] PATH_INFO: `/v3/projects/5480346b7e284479ab3ab16d176d570f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:04.967742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25621eba-9951-496f-bdae-4089adaa7689 None admin] RBAC: Authorizing `identity:get_project(project_id=5480346b7e284479ab3ab16d176d570f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:04.969205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25621eba-9951-496f-bdae-4089adaa7689 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:04.970956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 90/179] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:04 2022] GET /identity/v3/projects/5480346b7e284479ab3ab16d176d570f => generated 313 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:04.976947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb3d77-3dab-40cf-a2f7-69e0a06f1bad None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:05.133114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb3d77-3dab-40cf-a2f7-69e0a06f1bad None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:05.187906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abece3-2496-4ba9-83d4-077c0898452d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:05.188494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abece3-2496-4ba9-83d4-077c0898452d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:05.188775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abece3-2496-4ba9-83d4-077c0898452d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:05.189049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abece3-2496-4ba9-83d4-077c0898452d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:05.189890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abece3-2496-4ba9-83d4-077c0898452d None admin] RBAC: Adding query filter params (role.id=e87c07d8d70d46d89a2c4f7246edeff6, scope.project.id=5480346b7e284479ab3ab16d176d570f, user.id=7201ba55061e4bf4882217b64adb67b6) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:05.190200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abece3-2496-4ba9-83d4-077c0898452d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:05.191666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abece3-2496-4ba9-83d4-077c0898452d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:05.203756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 90/180] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:04 2022] GET /identity/v3/role_assignments?user.id=7201ba55061e4bf4882217b64adb67b6&scope.project.id=5480346b7e284479ab3ab16d176d570f&role.id=e87c07d8d70d46d89a2c4f7246edeff6 => generated 610 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:06.120498 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dbc6a-8a86-4b57-b1cc-0d3dacddb109 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:06.121026 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dbc6a-8a86-4b57-b1cc-0d3dacddb109 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:06.121104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dbc6a-8a86-4b57-b1cc-0d3dacddb109 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:06.122496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 91/181] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:39:06.127929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b2cb8-88ac-48bf-a776-14be2e349c5f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:06.128343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b2cb8-88ac-48bf-a776-14be2e349c5f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:06.128516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b2cb8-88ac-48bf-a776-14be2e349c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:06.134426 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:06.146965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-e37b2cb8-88ac-48bf-a776-14be2e349c5f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:06.148757 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e37b2cb8-88ac-48bf-a776-14be2e349c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:06.233716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 91/182] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:06.258955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dbc6a-8a86-4b57-b1cc-0d3dacddb109 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:06.373631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dbc6a-8a86-4b57-b1cc-0d3dacddb109 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:06.411210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:06.415776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18f5ff-77c4-4421-a292-93fecf7b326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:06.416784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18f5ff-77c4-4421-a292-93fecf7b326a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:06.417067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18f5ff-77c4-4421-a292-93fecf7b326a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:06.417638 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18f5ff-77c4-4421-a292-93fecf7b326a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:06.430554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18f5ff-77c4-4421-a292-93fecf7b326a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-8e18f5ff-77c4-4421-a292-93fecf7b326a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:06.438472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:06.439447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:06.440257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:06.440257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:06.440257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:06.440257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:06.440257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:06.440257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:06.440257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:06.440257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:06.440257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:06.440257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:06.440257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:06.440257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:06.443547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 92/183] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:39:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:06.446400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b2cb8-88ac-48bf-a776-14be2e349c5f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:06.451089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bfdba-4807-47db-b945-8dca3f7d4572 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:06.451375 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bfdba-4807-47db-b945-8dca3f7d4572 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:06.451652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bfdba-4807-47db-b945-8dca3f7d4572 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:06.453027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 93/184] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:39:06.458422 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454638ae-33aa-4ffb-a449-1a13a934882f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:06.459615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454638ae-33aa-4ffb-a449-1a13a934882f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:06.459908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454638ae-33aa-4ffb-a449-1a13a934882f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:06.478804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-454638ae-33aa-4ffb-a449-1a13a934882f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:06.480242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-454638ae-33aa-4ffb-a449-1a13a934882f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:06.556027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 94/185] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:06.570868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b2cb8-88ac-48bf-a776-14be2e349c5f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:06.579799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9385a-0112-44b7-b5b6-39c2fcfadba0 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:06.580181 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9385a-0112-44b7-b5b6-39c2fcfadba0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:06.580553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9385a-0112-44b7-b5b6-39c2fcfadba0 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:06.582085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 95/186] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:39:06.582649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9385a-0112-44b7-b5b6-39c2fcfadba0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:06.611329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:06.618252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880878e-1b18-4dd6-98fe-c6d63b0f9ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:06.619171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880878e-1b18-4dd6-98fe-c6d63b0f9ec4 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:06.619653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880878e-1b18-4dd6-98fe-c6d63b0f9ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:06.620239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880878e-1b18-4dd6-98fe-c6d63b0f9ec4 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:06.621490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880878e-1b18-4dd6-98fe-c6d63b0f9ec4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:06.621963 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880878e-1b18-4dd6-98fe-c6d63b0f9ec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:06.629986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880878e-1b18-4dd6-98fe-c6d63b0f9ec4 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:06.633865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 92/187] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:06 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:06.639748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c5e9b-bea4-4ad1-95a0-a831772b3592 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:06.640520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c5e9b-bea4-4ad1-95a0-a831772b3592 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:06.641065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c5e9b-bea4-4ad1-95a0-a831772b3592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:06.661711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-e24c5e9b-bea4-4ad1-95a0-a831772b3592 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:06.663741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e24c5e9b-bea4-4ad1-95a0-a831772b3592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:06.725461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9385a-0112-44b7-b5b6-39c2fcfadba0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:06.745356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 93/188] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:06.746855 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c5e9b-bea4-4ad1-95a0-a831772b3592 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:06.757745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:06.781237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b6f77-ae27-41b2-a939-b34de206a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:06.782039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b6f77-ae27-41b2-a939-b34de206a0bc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:06.782320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b6f77-ae27-41b2-a939-b34de206a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:06.782594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b6f77-ae27-41b2-a939-b34de206a0bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:06.794200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b6f77-ae27-41b2-a939-b34de206a0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e90b6f77-ae27-41b2-a939-b34de206a0bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:06.804528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:06.805781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:06.806558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:06.806558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:06.806558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:06.806558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:06.806558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:06.806558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:06.806558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:06.806558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:06.806558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:06.806558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:06.806558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:06.806558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:06.808190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 96/189] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:39:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:06.808190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b6f77-ae27-41b2-a939-b34de206a0bc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:06.922669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c5e9b-bea4-4ad1-95a0-a831772b3592 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:06.936198 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b6f77-ae27-41b2-a939-b34de206a0bc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:06.949425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:06.965227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f4cd1-22dc-4b8a-b7c4-48df850dfd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:06.966381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f4cd1-22dc-4b8a-b7c4-48df850dfd86 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:06.966740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f4cd1-22dc-4b8a-b7c4-48df850dfd86 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:06.967315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f4cd1-22dc-4b8a-b7c4-48df850dfd86 None admin] PATH_INFO: `/v3/users/7a40fca8538d4e87883ceda5e4d57c33` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:06.978928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424b0ed-4c69-472c-bc07-f1be64e4d336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:06.979790 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424b0ed-4c69-472c-bc07-f1be64e4d336 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:06.980075 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424b0ed-4c69-472c-bc07-f1be64e4d336 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:06.980379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424b0ed-4c69-472c-bc07-f1be64e4d336 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:06.981521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424b0ed-4c69-472c-bc07-f1be64e4d336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:06.983028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424b0ed-4c69-472c-bc07-f1be64e4d336 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:06.989244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 97/190] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:06 2022] GET /identity/v3/domains/default => generated 200 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:06.989381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f4cd1-22dc-4b8a-b7c4-48df850dfd86 None admin] RBAC: Authorizing `identity:get_user(user_id=7a40fca8538d4e87883ceda5e4d57c33)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:06.989481 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424b0ed-4c69-472c-bc07-f1be64e4d336 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:06.989553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f4cd1-22dc-4b8a-b7c4-48df850dfd86 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:06.989553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 94/191] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:06 2022] GET /identity/v3/users/7a40fca8538d4e87883ceda5e4d57c33 => generated 284 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:06.993814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f4cd1-22dc-4b8a-b7c4-48df850dfd86 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:07.051322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:07.119632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f4cd1-22dc-4b8a-b7c4-48df850dfd86 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:07.123920 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424b0ed-4c69-472c-bc07-f1be64e4d336 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:07.158799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abc386-36c5-4a8a-9837-8b4160fba155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:07.159366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2b861-c809-435a-b9e0-11854470b430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:07.159635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abc386-36c5-4a8a-9837-8b4160fba155 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:07.159798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abc386-36c5-4a8a-9837-8b4160fba155 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:07.159979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2b861-c809-435a-b9e0-11854470b430 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:07.160074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abc386-36c5-4a8a-9837-8b4160fba155 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:07.160261 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2b861-c809-435a-b9e0-11854470b430 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:07.160534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2b861-c809-435a-b9e0-11854470b430 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:07.160749 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abc386-36c5-4a8a-9837-8b4160fba155 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:07.161370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2b861-c809-435a-b9e0-11854470b430 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:07.161651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2b861-c809-435a-b9e0-11854470b430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:07.163357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abc386-36c5-4a8a-9837-8b4160fba155 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:07.163721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2b861-c809-435a-b9e0-11854470b430 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:07.173118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 98/192] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:06 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:07.173885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2b861-c809-435a-b9e0-11854470b430 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:07.273774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 95/193] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:39:06 2022] POST /identity/v3/users => generated 298 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:39:07.275004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abc386-36c5-4a8a-9837-8b4160fba155 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:07.299119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2b861-c809-435a-b9e0-11854470b430 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:07.301301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:07.331934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:07.356409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d628d-de57-41eb-b6e2-e2330bd52b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:07.357343 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d628d-de57-41eb-b6e2-e2330bd52b5a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:07.357776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d628d-de57-41eb-b6e2-e2330bd52b5a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:07.358053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d628d-de57-41eb-b6e2-e2330bd52b5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:07.417723 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abc386-36c5-4a8a-9837-8b4160fba155 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:07.458368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d628d-de57-41eb-b6e2-e2330bd52b5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:07.459459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837eb3c-2808-4514-9683-463974f137fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:07.459939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d628d-de57-41eb-b6e2-e2330bd52b5a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:07.460355 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837eb3c-2808-4514-9683-463974f137fd None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:07.460663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837eb3c-2808-4514-9683-463974f137fd None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:07.460936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837eb3c-2808-4514-9683-463974f137fd None admin] PATH_INFO: `/v3/users/7a40fca8538d4e87883ceda5e4d57c33` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:07.462191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837eb3c-2808-4514-9683-463974f137fd None admin] RBAC: Authorizing `identity:get_user(user_id=7a40fca8538d4e87883ceda5e4d57c33)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:07.463667 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837eb3c-2808-4514-9683-463974f137fd None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:07.465323 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 96/194] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:07 2022] GET /identity/v3/users/7a40fca8538d4e87883ceda5e4d57c33 => generated 284 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:07.466525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337a934-924e-47b1-adf7-71021e450d82 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:07.466806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337a934-924e-47b1-adf7-71021e450d82 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:07.471278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337a934-924e-47b1-adf7-71021e450d82 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:07.473507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 97/195] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:07 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:07.475135 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337a934-924e-47b1-adf7-71021e450d82 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:07.580779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 99/196] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:07 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:07.582100 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491ccd0-8f6c-42aa-a40d-4a4fc6892f7e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:07.582390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491ccd0-8f6c-42aa-a40d-4a4fc6892f7e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:07.582669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491ccd0-8f6c-42aa-a40d-4a4fc6892f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:07.597868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:07.600977 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7491ccd0-8f6c-42aa-a40d-4a4fc6892f7e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:07.602235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7491ccd0-8f6c-42aa-a40d-4a4fc6892f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:07.634384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337a934-924e-47b1-adf7-71021e450d82 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:07.642995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:07.677595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b6940-8b98-4e45-80e4-1dd306e1312b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:07.679099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b6940-8b98-4e45-80e4-1dd306e1312b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:07.679099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b6940-8b98-4e45-80e4-1dd306e1312b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:07.679099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b6940-8b98-4e45-80e4-1dd306e1312b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:07.690884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 100/197] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:07.691554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491ccd0-8f6c-42aa-a40d-4a4fc6892f7e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:07.786454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b6940-8b98-4e45-80e4-1dd306e1312b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:07.788946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b6940-8b98-4e45-80e4-1dd306e1312b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:07.815488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491ccd0-8f6c-42aa-a40d-4a4fc6892f7e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:07.854123 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169e4e7-a7ed-419a-a449-7ab1e2d37971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:07.855330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169e4e7-a7ed-419a-a449-7ab1e2d37971 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:07.855842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169e4e7-a7ed-419a-a449-7ab1e2d37971 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:07.856314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169e4e7-a7ed-419a-a449-7ab1e2d37971 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:07.857741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169e4e7-a7ed-419a-a449-7ab1e2d37971 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:07.859459 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169e4e7-a7ed-419a-a449-7ab1e2d37971 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:07.861418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 101/198] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:07 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:07.866079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:07.867869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169e4e7-a7ed-419a-a449-7ab1e2d37971 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:07.928446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 98/199] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:07 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:07.929858 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b6940-8b98-4e45-80e4-1dd306e1312b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:07.940192 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:07.986078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169e4e7-a7ed-419a-a449-7ab1e2d37971 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:08.044184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d224f72-85ea-46a0-93a7-daf66b20c5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:08.045014 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d224f72-85ea-46a0-93a7-daf66b20c5db None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:08.045328 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d224f72-85ea-46a0-93a7-daf66b20c5db None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:08.045602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d224f72-85ea-46a0-93a7-daf66b20c5db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:08.058862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d224f72-85ea-46a0-93a7-daf66b20c5db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:08.059452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b6940-8b98-4e45-80e4-1dd306e1312b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-9d224f72-85ea-46a0-93a7-daf66b20c5db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:08.072199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:08.073128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 102/200] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:08.073918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d224f72-85ea-46a0-93a7-daf66b20c5db None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:08.100817 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf439f1-0d41-4186-b8a0-76416eaecf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:08.101524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf439f1-0d41-4186-b8a0-76416eaecf62 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:08.101803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf439f1-0d41-4186-b8a0-76416eaecf62 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:08.102211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf439f1-0d41-4186-b8a0-76416eaecf62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:08.103306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf439f1-0d41-4186-b8a0-76416eaecf62 None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=77149f8497f3433c8a1c236067c9eb62, user.id=7a40fca8538d4e87883ceda5e4d57c33) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:08.103626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf439f1-0d41-4186-b8a0-76416eaecf62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:08.105119 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf439f1-0d41-4186-b8a0-76416eaecf62 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:08.111846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 99/201] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:07 2022] GET /identity/v3/role_assignments?user.id=7a40fca8538d4e87883ceda5e4d57c33&scope.project.id=77149f8497f3433c8a1c236067c9eb62&role.id=4f574961798246a4b12ad2276e67685d => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:08.112493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf439f1-0d41-4186-b8a0-76416eaecf62 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:08.141362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:08.196650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:08.229208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d224f72-85ea-46a0-93a7-daf66b20c5db None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:08.236634 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf439f1-0d41-4186-b8a0-76416eaecf62 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:08.286214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52efdae-a82c-402d-9245-2cb05d965467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:08.288779 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52efdae-a82c-402d-9245-2cb05d965467 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:08.289631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52efdae-a82c-402d-9245-2cb05d965467 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:08.290398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52efdae-a82c-402d-9245-2cb05d965467 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:08.292664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52efdae-a82c-402d-9245-2cb05d965467 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:08.293584 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52efdae-a82c-402d-9245-2cb05d965467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:08.296785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52efdae-a82c-402d-9245-2cb05d965467 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:08.311586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 100/202] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:08 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:08.323187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52efdae-a82c-402d-9245-2cb05d965467 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:08.330858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe8bc1-0c05-409c-b15f-819894755c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:08.332520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe8bc1-0c05-409c-b15f-819894755c74 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:08.332975 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe8bc1-0c05-409c-b15f-819894755c74 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:08.333507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe8bc1-0c05-409c-b15f-819894755c74 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:08.336568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe8bc1-0c05-409c-b15f-819894755c74 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:08.346887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe8bc1-0c05-409c-b15f-819894755c74 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:08.346887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 103/203] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:08 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:08.350258 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe8bc1-0c05-409c-b15f-819894755c74 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:08.500921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52efdae-a82c-402d-9245-2cb05d965467 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:08.509278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe8bc1-0c05-409c-b15f-819894755c74 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:08.538097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:08.539348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:08.560496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183490a-3453-4ab6-8468-714ba4a7a741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:08.560496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183490a-3453-4ab6-8468-714ba4a7a741 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:08.560496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183490a-3453-4ab6-8468-714ba4a7a741 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:08.560496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183490a-3453-4ab6-8468-714ba4a7a741 None admin] PATH_INFO: `/v3/users/dbda967853b6446480e1a026eab76ae0` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:08.560961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191d427-4f9c-4c29-869a-d1f0b847cc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:08.560961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191d427-4f9c-4c29-869a-d1f0b847cc68 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:08.560961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191d427-4f9c-4c29-869a-d1f0b847cc68 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:08.560961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191d427-4f9c-4c29-869a-d1f0b847cc68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:08.561213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191d427-4f9c-4c29-869a-d1f0b847cc68 None admin] RBAC: Adding query filter params (role.id=582f59fc05ce474287c6fdc503f9ac82, scope.project.id=77149f8497f3433c8a1c236067c9eb62, user.id=7a40fca8538d4e87883ceda5e4d57c33) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:08.561213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191d427-4f9c-4c29-869a-d1f0b847cc68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:08.561213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191d427-4f9c-4c29-869a-d1f0b847cc68 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:08.562757 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183490a-3453-4ab6-8468-714ba4a7a741 None admin] RBAC: Authorizing `identity:get_user(user_id=dbda967853b6446480e1a026eab76ae0)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:08.564244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183490a-3453-4ab6-8468-714ba4a7a741 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:08.566272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 101/204] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:08 2022] GET /identity/v3/users/dbda967853b6446480e1a026eab76ae0 => generated 290 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:08.575385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 104/205] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:08 2022] GET /identity/v3/role_assignments?user.id=7a40fca8538d4e87883ceda5e4d57c33&scope.project.id=77149f8497f3433c8a1c236067c9eb62&role.id=582f59fc05ce474287c6fdc503f9ac82 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:08.577932 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183490a-3453-4ab6-8468-714ba4a7a741 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:08.736607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183490a-3453-4ab6-8468-714ba4a7a741 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:08.797520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4d29e-1d39-4755-bd6d-d26fd15a285d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:08.798327 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4d29e-1d39-4755-bd6d-d26fd15a285d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:08.798609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4d29e-1d39-4755-bd6d-d26fd15a285d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:08.798916 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4d29e-1d39-4755-bd6d-d26fd15a285d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:08.921498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:08.926733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4d29e-1d39-4755-bd6d-d26fd15a285d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:08.932651 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4d29e-1d39-4755-bd6d-d26fd15a285d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:09.076058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 102/206] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:08 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:09.084361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191d427-4f9c-4c29-869a-d1f0b847cc68 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:09.252023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191d427-4f9c-4c29-869a-d1f0b847cc68 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:09.309429 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368da868-f25c-44ac-812f-1fc1772514ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:09.310321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368da868-f25c-44ac-812f-1fc1772514ec None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:09.310596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368da868-f25c-44ac-812f-1fc1772514ec None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:09.310889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368da868-f25c-44ac-812f-1fc1772514ec None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:09.315693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368da868-f25c-44ac-812f-1fc1772514ec None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:09.315693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368da868-f25c-44ac-812f-1fc1772514ec None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:09.317591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 105/207] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:09 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:09.324176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4d29e-1d39-4755-bd6d-d26fd15a285d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:09.493065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:09.497642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4d29e-1d39-4755-bd6d-d26fd15a285d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:09.545197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecad46d-a5e8-49fa-980c-39964a0bebf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:09.545786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecad46d-a5e8-49fa-980c-39964a0bebf0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:09.546063 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecad46d-a5e8-49fa-980c-39964a0bebf0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:09.546333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecad46d-a5e8-49fa-980c-39964a0bebf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:09.547190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecad46d-a5e8-49fa-980c-39964a0bebf0 None admin] RBAC: Adding query filter params (role.id=6d1a0644f0bf45b4aa0f9fd0b3843261, scope.project.id=8611e940b0fa4f8d910441467b31bdce, user.id=dbda967853b6446480e1a026eab76ae0) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:09.547497 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecad46d-a5e8-49fa-980c-39964a0bebf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:09.548941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecad46d-a5e8-49fa-980c-39964a0bebf0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:09.559998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 103/208] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:09 2022] GET /identity/v3/role_assignments?user.id=dbda967853b6446480e1a026eab76ae0&scope.project.id=8611e940b0fa4f8d910441467b31bdce&role.id=6d1a0644f0bf45b4aa0f9fd0b3843261 => generated 610 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:10.105190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39de5f-0b59-48a5-a30f-7e9f428a7d0c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:10.105190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39de5f-0b59-48a5-a30f-7e9f428a7d0c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:10.105485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39de5f-0b59-48a5-a30f-7e9f428a7d0c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:10.107379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 106/209] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:10.113458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b16952-ed79-4312-8f6c-a0186dea6f88 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:10.113852 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b16952-ed79-4312-8f6c-a0186dea6f88 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:10.114229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b16952-ed79-4312-8f6c-a0186dea6f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:10.137070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b3b16952-ed79-4312-8f6c-a0186dea6f88 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:10.138277 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b3b16952-ed79-4312-8f6c-a0186dea6f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:10.235849 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 104/210] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:10 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:10.269522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39de5f-0b59-48a5-a30f-7e9f428a7d0c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:10.348017 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:10.394223 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39de5f-0b59-48a5-a30f-7e9f428a7d0c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:10.431018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b021c-d250-4623-aac3-a27114f0568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:10.431828 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b021c-d250-4623-aac3-a27114f0568b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:10.432103 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b021c-d250-4623-aac3-a27114f0568b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:10.432370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b021c-d250-4623-aac3-a27114f0568b None admin] PATH_INFO: `/v3/roles/member` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:10.444308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b021c-d250-4623-aac3-a27114f0568b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-450b021c-d250-4623-aac3-a27114f0568b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:10.453213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:10.454118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:10.454932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 107/211] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:10.460931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b16952-ed79-4312-8f6c-a0186dea6f88 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:10.465365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada53c50-98b5-4815-80b8-7770162a7419 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:10.465652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada53c50-98b5-4815-80b8-7770162a7419 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:10.465927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada53c50-98b5-4815-80b8-7770162a7419 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:10.467620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 108/212] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:39:10.472730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6079dd4-cbc8-4e5a-8320-143295203687 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:10.473020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6079dd4-cbc8-4e5a-8320-143295203687 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:10.473303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6079dd4-cbc8-4e5a-8320-143295203687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:10.492522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b6079dd4-cbc8-4e5a-8320-143295203687 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:10.493726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b6079dd4-cbc8-4e5a-8320-143295203687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:10.571547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:10.577274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 109/213] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:10 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:10.578165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b16952-ed79-4312-8f6c-a0186dea6f88 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:10.603950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6079dd4-cbc8-4e5a-8320-143295203687 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:10.621189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12846b4-33f1-4d95-b1ed-12a3a5795a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:10.621871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12846b4-33f1-4d95-b1ed-12a3a5795a6b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:10.622151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12846b4-33f1-4d95-b1ed-12a3a5795a6b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:10.622425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12846b4-33f1-4d95-b1ed-12a3a5795a6b None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:10.623228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12846b4-33f1-4d95-b1ed-12a3a5795a6b None admin] RBAC: Adding query filter params (name=member) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:10.623536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12846b4-33f1-4d95-b1ed-12a3a5795a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:10.624924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12846b4-33f1-4d95-b1ed-12a3a5795a6b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:10.636700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 105/214] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:10 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:10.642201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12846b4-33f1-4d95-b1ed-12a3a5795a6b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:10.673395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:10.743910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6079dd4-cbc8-4e5a-8320-143295203687 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:10.760048 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12846b4-33f1-4d95-b1ed-12a3a5795a6b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:10.818350 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261d472-4005-407f-ab17-46567747f96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:10.819066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bc302-e345-4bb3-b482-b3459ea0a89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:10.819492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261d472-4005-407f-ab17-46567747f96d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:10.819787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261d472-4005-407f-ab17-46567747f96d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:10.820260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261d472-4005-407f-ab17-46567747f96d None admin] PATH_INFO: `/v3/users/227f36fe6ea8435ca96db65897793e10` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:10.820786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bc302-e345-4bb3-b482-b3459ea0a89a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:10.821254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bc302-e345-4bb3-b482-b3459ea0a89a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:10.821744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bc302-e345-4bb3-b482-b3459ea0a89a None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:10.823454 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bc302-e345-4bb3-b482-b3459ea0a89a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:10.826021 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bc302-e345-4bb3-b482-b3459ea0a89a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:10.832679 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 110/215] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:10 2022] GET /identity/v3/domains/default => generated 200 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:10.833219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bc302-e345-4bb3-b482-b3459ea0a89a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:10.841125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261d472-4005-407f-ab17-46567747f96d None admin] RBAC: Authorizing `identity:get_user(user_id=227f36fe6ea8435ca96db65897793e10)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:10.842605 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261d472-4005-407f-ab17-46567747f96d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:10.846911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 106/216] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:10 2022] GET /identity/v3/users/227f36fe6ea8435ca96db65897793e10 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:10.849826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261d472-4005-407f-ab17-46567747f96d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:10.869282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:10.963768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261d472-4005-407f-ab17-46567747f96d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:10.970320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bc302-e345-4bb3-b482-b3459ea0a89a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:11.000736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:11.024993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a311172-3a63-4170-8030-b1a81b6d31de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:11.025398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14502f-2949-46a7-9848-493ba5871704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:11.025917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a311172-3a63-4170-8030-b1a81b6d31de None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:11.026170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14502f-2949-46a7-9848-493ba5871704 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:11.026418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a311172-3a63-4170-8030-b1a81b6d31de None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:11.026584 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14502f-2949-46a7-9848-493ba5871704 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:11.026858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14502f-2949-46a7-9848-493ba5871704 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:11.027005 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a311172-3a63-4170-8030-b1a81b6d31de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:11.027594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14502f-2949-46a7-9848-493ba5871704 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:11.029099 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14502f-2949-46a7-9848-493ba5871704 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:11.109090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a311172-3a63-4170-8030-b1a81b6d31de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:11.110409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a311172-3a63-4170-8030-b1a81b6d31de None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:11.127839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:11.139051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 111/217] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:39:10 2022] POST /identity/v3/users => generated 298 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:39:11.139051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50e143-d69b-4342-accc-b24da1c4de39 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:11.139051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50e143-d69b-4342-accc-b24da1c4de39 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:11.139051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50e143-d69b-4342-accc-b24da1c4de39 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:11.139051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 112/218] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:11.149407 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14249d8-86ef-402e-872d-86578280bf56 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:11.149690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14249d8-86ef-402e-872d-86578280bf56 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:11.149969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14249d8-86ef-402e-872d-86578280bf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:11.170417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e14249d8-86ef-402e-872d-86578280bf56 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:11.171542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e14249d8-86ef-402e-872d-86578280bf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:11.263073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 113/219] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:11.272871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 107/220] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:10 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:39:11.278950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14249d8-86ef-402e-872d-86578280bf56 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:11.287252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a311172-3a63-4170-8030-b1a81b6d31de None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:11.416696 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a311172-3a63-4170-8030-b1a81b6d31de None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:11.425598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14249d8-86ef-402e-872d-86578280bf56 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:11.428402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:11.435772 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:11.460307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234e441-8cb2-48a7-af85-1c13c07d2b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:11.461701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234e441-8cb2-48a7-af85-1c13c07d2b42 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:11.462238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234e441-8cb2-48a7-af85-1c13c07d2b42 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:11.462403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234e441-8cb2-48a7-af85-1c13c07d2b42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:11.470521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72444682-3594-478e-870a-6feb95f77a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:11.471433 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72444682-3594-478e-870a-6feb95f77a31 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:11.471711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72444682-3594-478e-870a-6feb95f77a31 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:11.472299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72444682-3594-478e-870a-6feb95f77a31 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:11.474580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72444682-3594-478e-870a-6feb95f77a31 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:11.476929 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72444682-3594-478e-870a-6feb95f77a31 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:11.479569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234e441-8cb2-48a7-af85-1c13c07d2b42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:11.483836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 114/221] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:11 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:11.484943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204553a-42c5-48a4-9bb2-f0aa8f8577c2 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:11.485293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204553a-42c5-48a4-9bb2-f0aa8f8577c2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:11.486756 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204553a-42c5-48a4-9bb2-f0aa8f8577c2 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:11.487081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 115/222] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d234e441-8cb2-48a7-af85-1c13c07d2b42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:11.487400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:11.488218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:11.489169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:11.489169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:11.489169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:11.489169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:11.489169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:11.489169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:11.489169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:11.489169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:11.489169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:11.489169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:11.489169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:11.489169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:11.490615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204553a-42c5-48a4-9bb2-f0aa8f8577c2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:11.490698 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 108/223] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:39:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:11.495642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fb788-22f8-44d4-88df-3066f91b28eb None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:11.495923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fb788-22f8-44d4-88df-3066f91b28eb None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:11.496248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fb788-22f8-44d4-88df-3066f91b28eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:11.520523 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-521fb788-22f8-44d4-88df-3066f91b28eb None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:11.521798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-521fb788-22f8-44d4-88df-3066f91b28eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:11.615638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 109/224] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:11.616708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fb788-22f8-44d4-88df-3066f91b28eb None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:11.618302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204553a-42c5-48a4-9bb2-f0aa8f8577c2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:11.665731 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6e3b6-ae91-4fee-bcea-03865866493b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:11.667666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6e3b6-ae91-4fee-bcea-03865866493b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:11.668109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6e3b6-ae91-4fee-bcea-03865866493b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:11.668390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6e3b6-ae91-4fee-bcea-03865866493b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:11.669685 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6e3b6-ae91-4fee-bcea-03865866493b None admin] RBAC: Adding query filter params (role.id=e87c07d8d70d46d89a2c4f7246edeff6, scope.project.id=77149f8497f3433c8a1c236067c9eb62, user.id=227f36fe6ea8435ca96db65897793e10) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:11.669998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6e3b6-ae91-4fee-bcea-03865866493b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:11.671810 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6e3b6-ae91-4fee-bcea-03865866493b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:11.680763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 116/225] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:11 2022] GET /identity/v3/role_assignments?user.id=227f36fe6ea8435ca96db65897793e10&scope.project.id=77149f8497f3433c8a1c236067c9eb62&role.id=e87c07d8d70d46d89a2c4f7246edeff6 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:11.681455 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6e3b6-ae91-4fee-bcea-03865866493b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:11.774921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:11.808847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fb788-22f8-44d4-88df-3066f91b28eb None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:11.816930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:11.864510 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6e3b6-ae91-4fee-bcea-03865866493b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:11.907583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9fb7a-c71f-462a-9889-faf0c61c9bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:11.908284 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9fb7a-c71f-462a-9889-faf0c61c9bc4 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:11.908567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9fb7a-c71f-462a-9889-faf0c61c9bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:11.908843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9fb7a-c71f-462a-9889-faf0c61c9bc4 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:11.909670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9fb7a-c71f-462a-9889-faf0c61c9bc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:11.909982 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9fb7a-c71f-462a-9889-faf0c61c9bc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:11.912072 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9fb7a-c71f-462a-9889-faf0c61c9bc4 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:11.924566 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea115c65-ad71-4d5d-bdc0-fd0db0db77fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:11.924769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 110/226] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:11 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:11.925394 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea115c65-ad71-4d5d-bdc0-fd0db0db77fa None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:11.925671 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea115c65-ad71-4d5d-bdc0-fd0db0db77fa None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:11.925943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea115c65-ad71-4d5d-bdc0-fd0db0db77fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:11.931211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9fb7a-c71f-462a-9889-faf0c61c9bc4 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:11.960169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea115c65-ad71-4d5d-bdc0-fd0db0db77fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-ea115c65-ad71-4d5d-bdc0-fd0db0db77fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:11.968883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:11.969778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:11.970564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 117/227] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:39:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:11.980108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea115c65-ad71-4d5d-bdc0-fd0db0db77fa None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:12.048352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9fb7a-c71f-462a-9889-faf0c61c9bc4 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:12.087518 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ed3b6-462b-4432-8a6b-a0e655a0977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:12.088405 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ed3b6-462b-4432-8a6b-a0e655a0977a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:12.088686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ed3b6-462b-4432-8a6b-a0e655a0977a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:12.089123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ed3b6-462b-4432-8a6b-a0e655a0977a None admin] PATH_INFO: `/v3/users/7a40fca8538d4e87883ceda5e4d57c33` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:12.090494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ed3b6-462b-4432-8a6b-a0e655a0977a None admin] RBAC: Authorizing `identity:get_user(user_id=7a40fca8538d4e87883ceda5e4d57c33)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:12.096927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ed3b6-462b-4432-8a6b-a0e655a0977a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:12.105653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 111/228] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:11 2022] GET /identity/v3/users/7a40fca8538d4e87883ceda5e4d57c33 => generated 284 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:12.108355 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ed3b6-462b-4432-8a6b-a0e655a0977a None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:12.147417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:12.182245 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:12.195756 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea115c65-ad71-4d5d-bdc0-fd0db0db77fa None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:12.252278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59b24d-f2af-4d5e-897b-81509c00b8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:12.252955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59b24d-f2af-4d5e-897b-81509c00b8f4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:12.253363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59b24d-f2af-4d5e-897b-81509c00b8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:12.253674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59b24d-f2af-4d5e-897b-81509c00b8f4 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:12.254532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59b24d-f2af-4d5e-897b-81509c00b8f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:12.254860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59b24d-f2af-4d5e-897b-81509c00b8f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:12.257015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59b24d-f2af-4d5e-897b-81509c00b8f4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:12.265656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ed3b6-462b-4432-8a6b-a0e655a0977a None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:12.266121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 118/229] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:11 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:12.272962 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59b24d-f2af-4d5e-897b-81509c00b8f4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:12.333160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb6601-a4f4-47e1-ab7e-f9d65602e8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:12.334640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb6601-a4f4-47e1-ab7e-f9d65602e8ce None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:12.334948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb6601-a4f4-47e1-ab7e-f9d65602e8ce None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:12.335282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb6601-a4f4-47e1-ab7e-f9d65602e8ce None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:12.339586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb6601-a4f4-47e1-ab7e-f9d65602e8ce None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:12.341272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb6601-a4f4-47e1-ab7e-f9d65602e8ce None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:12.343028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 112/230] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:12 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:12.349813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb6601-a4f4-47e1-ab7e-f9d65602e8ce None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:12.431452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59b24d-f2af-4d5e-897b-81509c00b8f4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:12.471668 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fa419-0f48-47fe-8291-a2695918368b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:12.472951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fa419-0f48-47fe-8291-a2695918368b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:12.473573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fa419-0f48-47fe-8291-a2695918368b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:12.474249 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fa419-0f48-47fe-8291-a2695918368b None admin] PATH_INFO: `/v3/users/7a40fca8538d4e87883ceda5e4d57c33` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:12.475984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fa419-0f48-47fe-8291-a2695918368b None admin] RBAC: Authorizing `identity:get_user(user_id=7a40fca8538d4e87883ceda5e4d57c33)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:12.477659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fa419-0f48-47fe-8291-a2695918368b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:12.477984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb6601-a4f4-47e1-ab7e-f9d65602e8ce None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:12.479617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:12.480617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 119/231] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:12 2022] GET /identity/v3/users/7a40fca8538d4e87883ceda5e4d57c33 => generated 284 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:12.486192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fa419-0f48-47fe-8291-a2695918368b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:12.518846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c08a3-ca66-4c62-9191-0ef4415a326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:12.518846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c08a3-ca66-4c62-9191-0ef4415a326e None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:12.518846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c08a3-ca66-4c62-9191-0ef4415a326e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:12.518846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c08a3-ca66-4c62-9191-0ef4415a326e None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62/users/7a40fca8538d4e87883ceda5e4d57c33/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:12.519275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c08a3-ca66-4c62-9191-0ef4415a326e None admin] RBAC: Authorizing `identity:create_grant(project_id=77149f8497f3433c8a1c236067c9eb62, user_id=7a40fca8538d4e87883ceda5e4d57c33, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:12.519928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c08a3-ca66-4c62-9191-0ef4415a326e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:12.540101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:12.584135 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 113/232] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:39:12 2022] PUT /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62/users/7a40fca8538d4e87883ceda5e4d57c33/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:39:12.643670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fa419-0f48-47fe-8291-a2695918368b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:12.704048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe8f36-3b79-4315-a6a7-3b1dd1065be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:12.705056 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe8f36-3b79-4315-a6a7-3b1dd1065be4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:12.705384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe8f36-3b79-4315-a6a7-3b1dd1065be4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:12.705676 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe8f36-3b79-4315-a6a7-3b1dd1065be4 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:12.709956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe8f36-3b79-4315-a6a7-3b1dd1065be4 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:12.711710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe8f36-3b79-4315-a6a7-3b1dd1065be4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:12.713929 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 120/233] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:12 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:12.720042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c08a3-ca66-4c62-9191-0ef4415a326e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:12.867196 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c08a3-ca66-4c62-9191-0ef4415a326e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:12.912142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92bdcd-ebb0-41be-aea3-e4d8a4535c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:12.913260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92bdcd-ebb0-41be-aea3-e4d8a4535c2d None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:12.913558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92bdcd-ebb0-41be-aea3-e4d8a4535c2d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:12.913832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92bdcd-ebb0-41be-aea3-e4d8a4535c2d None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62/users/7a40fca8538d4e87883ceda5e4d57c33/roles/582f59fc05ce474287c6fdc503f9ac82` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:12.916148 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92bdcd-ebb0-41be-aea3-e4d8a4535c2d None admin] RBAC: Authorizing `identity:create_grant(project_id=77149f8497f3433c8a1c236067c9eb62, user_id=7a40fca8538d4e87883ceda5e4d57c33, role_id=582f59fc05ce474287c6fdc503f9ac82)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:12.917870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92bdcd-ebb0-41be-aea3-e4d8a4535c2d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:13.001111 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 114/234] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:39:12 2022] PUT /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62/users/7a40fca8538d4e87883ceda5e4d57c33/roles/582f59fc05ce474287c6fdc503f9ac82 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:39:14.120452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962b86b-a04d-4627-8ed2-353c5d073e80 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:14.121113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962b86b-a04d-4627-8ed2-353c5d073e80 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:14.121113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962b86b-a04d-4627-8ed2-353c5d073e80 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:14.122568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 121/235] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:14.128100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d79f9-8d89-4ea3-92b6-b6871efb385c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:14.128703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d79f9-8d89-4ea3-92b6-b6871efb385c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:14.128703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d79f9-8d89-4ea3-92b6-b6871efb385c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:14.148589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f08d79f9-8d89-4ea3-92b6-b6871efb385c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:14.149818 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f08d79f9-8d89-4ea3-92b6-b6871efb385c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:14.168966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:14.195696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93555f9d-bd8e-4802-9a3d-2acecfa9a168 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:14.195696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93555f9d-bd8e-4802-9a3d-2acecfa9a168 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:14.195696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93555f9d-bd8e-4802-9a3d-2acecfa9a168 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:14.197819 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 122/236] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:14.205238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09fea5-5c0c-4353-b623-22c9fe0675bf None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:14.205512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09fea5-5c0c-4353-b623-22c9fe0675bf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:14.205792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09fea5-5c0c-4353-b623-22c9fe0675bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:14.230527 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6e09fea5-5c0c-4353-b623-22c9fe0675bf None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:14.232463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6e09fea5-5c0c-4353-b623-22c9fe0675bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:14.252593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 115/237] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:14.336928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d79f9-8d89-4ea3-92b6-b6871efb385c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:14.346848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 123/238] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:14.367432 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09fea5-5c0c-4353-b623-22c9fe0675bf None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:14.401969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:14.508545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09fea5-5c0c-4353-b623-22c9fe0675bf None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:14.554107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9f74-c7c6-4f69-b7b4-a4d2a59bfdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:14.554958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9f74-c7c6-4f69-b7b4-a4d2a59bfdaa None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:14.555427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9f74-c7c6-4f69-b7b4-a4d2a59bfdaa None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:14.555720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d9f74-c7c6-4f69-b7b4-a4d2a59bfdaa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:14.575001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d9f74-c7c6-4f69-b7b4-a4d2a59bfdaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-ad0d9f74-c7c6-4f69-b7b4-a4d2a59bfdaa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:14.592360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:14.593304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:14.594069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:14.594069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:14.594069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:14.594069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:14.594069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:14.594069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:14.594069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:14.594069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:14.594069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:14.594069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:14.594069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:14.594069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:14.599872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 124/239] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:14.601927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9f74-c7c6-4f69-b7b4-a4d2a59bfdaa None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:14.689215 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d79f9-8d89-4ea3-92b6-b6871efb385c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:14.705676 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:14.726056 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d9f74-c7c6-4f69-b7b4-a4d2a59bfdaa None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:14.742463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ab6bc-8856-47d2-9db5-fddc55f31989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:14.744050 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ab6bc-8856-47d2-9db5-fddc55f31989 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:14.744346 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ab6bc-8856-47d2-9db5-fddc55f31989 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:14.744622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ab6bc-8856-47d2-9db5-fddc55f31989 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:14.746481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ab6bc-8856-47d2-9db5-fddc55f31989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:14.748015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ab6bc-8856-47d2-9db5-fddc55f31989 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:14.750246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 116/240] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:14 2022] GET /identity/v3/domains/default => generated 200 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:14.754024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6779f-5ced-426a-8764-838f204ae6dd None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:14.754631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6779f-5ced-426a-8764-838f204ae6dd None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:14.754943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6779f-5ced-426a-8764-838f204ae6dd None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:14.758960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 117/241] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:14 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:14.759870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6779f-5ced-426a-8764-838f204ae6dd None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:14.771682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db143ef-bd24-4b4c-812b-4e94c67df070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:14.772367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db143ef-bd24-4b4c-812b-4e94c67df070 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:14.772654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db143ef-bd24-4b4c-812b-4e94c67df070 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:14.772965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db143ef-bd24-4b4c-812b-4e94c67df070 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:14.773743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db143ef-bd24-4b4c-812b-4e94c67df070 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:14.774050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db143ef-bd24-4b4c-812b-4e94c67df070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:14.775621 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db143ef-bd24-4b4c-812b-4e94c67df070 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:14.786497 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 125/242] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:14 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:14.789717 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb9048-d076-4682-8181-e0929a2d087c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:14.790251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb9048-d076-4682-8181-e0929a2d087c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:14.790724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb9048-d076-4682-8181-e0929a2d087c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:14.798046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:14.817368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-93bb9048-d076-4682-8181-e0929a2d087c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:14.823382 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-93bb9048-d076-4682-8181-e0929a2d087c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:14.906179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 126/243] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:14.907520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb9048-d076-4682-8181-e0929a2d087c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:14.925084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6779f-5ced-426a-8764-838f204ae6dd None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:14.982203 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4e5e4-f690-4c5a-84e0-0f84a29a2d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:14.982849 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4e5e4-f690-4c5a-84e0-0f84a29a2d99 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:14.983132 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4e5e4-f690-4c5a-84e0-0f84a29a2d99 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:14.983680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4e5e4-f690-4c5a-84e0-0f84a29a2d99 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:14.984422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4e5e4-f690-4c5a-84e0-0f84a29a2d99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:14.986309 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4e5e4-f690-4c5a-84e0-0f84a29a2d99 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:15.020028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:15.049791 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb9048-d076-4682-8181-e0929a2d087c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:15.087976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a788c-712a-4938-9d9c-ac139359d35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:15.088874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a788c-712a-4938-9d9c-ac139359d35f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:15.089149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a788c-712a-4938-9d9c-ac139359d35f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:15.089422 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a788c-712a-4938-9d9c-ac139359d35f None admin] PATH_INFO: `/v3/users/fca02058f3cd469688130fca81d450bd` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:15.115231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 118/244] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:39:14 2022] POST /identity/v3/users => generated 294 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:39:15.115231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4e5e4-f690-4c5a-84e0-0f84a29a2d99 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:15.122134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a788c-712a-4938-9d9c-ac139359d35f None admin] RBAC: Authorizing `identity:get_user(user_id=fca02058f3cd469688130fca81d450bd)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:15.123780 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a788c-712a-4938-9d9c-ac139359d35f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:15.132476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 127/245] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:14 2022] GET /identity/v3/users/fca02058f3cd469688130fca81d450bd => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:15.133628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a788c-712a-4938-9d9c-ac139359d35f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:15.261951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a788c-712a-4938-9d9c-ac139359d35f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:15.302415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e0acd-140b-4dbf-b126-aed8af92f127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:15.303216 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e0acd-140b-4dbf-b126-aed8af92f127 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:15.303506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e0acd-140b-4dbf-b126-aed8af92f127 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:15.303784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e0acd-140b-4dbf-b126-aed8af92f127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:15.306746 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:15.319211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:15.357678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4e5e4-f690-4c5a-84e0-0f84a29a2d99 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:15.413678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5a39b-bc83-4ba7-992d-285f788a3c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:15.414991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e0acd-140b-4dbf-b126-aed8af92f127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:15.415128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5a39b-bc83-4ba7-992d-285f788a3c61 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:15.415291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5a39b-bc83-4ba7-992d-285f788a3c61 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:15.415712 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5a39b-bc83-4ba7-992d-285f788a3c61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:15.417265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e0acd-140b-4dbf-b126-aed8af92f127 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:15.430334 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5a39b-bc83-4ba7-992d-285f788a3c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d5b5a39b-bc83-4ba7-992d-285f788a3c61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:15.439077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:15.440023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:15.440838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:15.440838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:15.440838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:15.440838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:15.440838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:15.440838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:15.440838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:15.440838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:15.440838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:15.440838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:15.440838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:15.440838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:15.443896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 119/246] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:15.445467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4f464-771a-43df-8b40-3863ee2fb920 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:15.445900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4f464-771a-43df-8b40-3863ee2fb920 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:15.446335 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4f464-771a-43df-8b40-3863ee2fb920 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:15.448071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 120/247] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:15.451267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4f464-771a-43df-8b40-3863ee2fb920 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:15.562966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 128/248] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:15 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:15.564570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab9b76-bea5-45bd-af79-d8a1d49d0820 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:15.565032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab9b76-bea5-45bd-af79-d8a1d49d0820 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:15.565472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab9b76-bea5-45bd-af79-d8a1d49d0820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:15.583884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bfab9b76-bea5-45bd-af79-d8a1d49d0820 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:15.585545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bfab9b76-bea5-45bd-af79-d8a1d49d0820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:15.600099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4f464-771a-43df-8b40-3863ee2fb920 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:15.620708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:15.643115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65c037-09f2-4356-99d8-956ec8713be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:15.644051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65c037-09f2-4356-99d8-956ec8713be0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:15.644510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65c037-09f2-4356-99d8-956ec8713be0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:15.644859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65c037-09f2-4356-99d8-956ec8713be0 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:15.645760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65c037-09f2-4356-99d8-956ec8713be0 None admin] RBAC: Adding query filter params (name=member) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:15.646143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65c037-09f2-4356-99d8-956ec8713be0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:15.648340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65c037-09f2-4356-99d8-956ec8713be0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:15.651419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:15.659996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 121/249] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:15 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:15.660943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65c037-09f2-4356-99d8-956ec8713be0 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:15.683829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 129/250] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:15.685134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab9b76-bea5-45bd-af79-d8a1d49d0820 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:15.828877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab9b76-bea5-45bd-af79-d8a1d49d0820 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:15.853862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65c037-09f2-4356-99d8-956ec8713be0 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:15.872946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dac245-9499-4533-8927-612c4766b0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:15.875545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dac245-9499-4533-8927-612c4766b0b5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:15.876079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dac245-9499-4533-8927-612c4766b0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:15.876636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dac245-9499-4533-8927-612c4766b0b5 None admin] PATH_INFO: `/v3/users/227f36fe6ea8435ca96db65897793e10` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:15.879114 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dac245-9499-4533-8927-612c4766b0b5 None admin] RBAC: Authorizing `identity:get_user(user_id=227f36fe6ea8435ca96db65897793e10)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:15.882288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dac245-9499-4533-8927-612c4766b0b5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:15.887122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 130/251] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:15 2022] GET /identity/v3/users/227f36fe6ea8435ca96db65897793e10 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:15.888191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dac245-9499-4533-8927-612c4766b0b5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:15.928652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e077db-a86b-46e5-97a3-855d9742a60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:15.929546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e077db-a86b-46e5-97a3-855d9742a60c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:15.929825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e077db-a86b-46e5-97a3-855d9742a60c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:15.930099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e077db-a86b-46e5-97a3-855d9742a60c None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:15.932622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e077db-a86b-46e5-97a3-855d9742a60c None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:15.934103 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e077db-a86b-46e5-97a3-855d9742a60c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:15.936116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 122/252] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:15 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:15.936890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e077db-a86b-46e5-97a3-855d9742a60c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:15.949943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:16.049236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:16.061143 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dac245-9499-4533-8927-612c4766b0b5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:16.100429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e077db-a86b-46e5-97a3-855d9742a60c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:16.115369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4afab-223f-4c72-99fb-da8199d6fa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:16.116721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4afab-223f-4c72-99fb-da8199d6fa64 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:16.117106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4afab-223f-4c72-99fb-da8199d6fa64 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:16.117487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4afab-223f-4c72-99fb-da8199d6fa64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:16.131699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4afab-223f-4c72-99fb-da8199d6fa64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-62a4afab-223f-4c72-99fb-da8199d6fa64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:16.142042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:16.142996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:16.143784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 131/253] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:39:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:16.144745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4afab-223f-4c72-99fb-da8199d6fa64 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:16.160333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5eee6-8b1a-465d-94c8-7d28b9ef44a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:16.162651 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5eee6-8b1a-465d-94c8-7d28b9ef44a4 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:16.163177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5eee6-8b1a-465d-94c8-7d28b9ef44a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:16.163751 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5eee6-8b1a-465d-94c8-7d28b9ef44a4 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:16.166134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5eee6-8b1a-465d-94c8-7d28b9ef44a4 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:16.169286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5eee6-8b1a-465d-94c8-7d28b9ef44a4 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:16.171622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 123/254] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:15 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:16.173145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc5373-40c4-4f39-9d88-a3081208aa04 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:16.173433 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc5373-40c4-4f39-9d88-a3081208aa04 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:16.173708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc5373-40c4-4f39-9d88-a3081208aa04 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:16.175134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 124/255] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:16.175754 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc5373-40c4-4f39-9d88-a3081208aa04 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:16.265074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:16.275202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4afab-223f-4c72-99fb-da8199d6fa64 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:16.313694 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879aeb24-b688-44d0-baa8-ce843f6840bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:16.314351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879aeb24-b688-44d0-baa8-ce843f6840bb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:16.314628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879aeb24-b688-44d0-baa8-ce843f6840bb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:16.314943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879aeb24-b688-44d0-baa8-ce843f6840bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:16.315783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879aeb24-b688-44d0-baa8-ce843f6840bb None admin] RBAC: Adding query filter params (role.id=6d1a0644f0bf45b4aa0f9fd0b3843261, scope.project.id=77149f8497f3433c8a1c236067c9eb62, user.id=fca02058f3cd469688130fca81d450bd) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:16.316096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879aeb24-b688-44d0-baa8-ce843f6840bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:16.317741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879aeb24-b688-44d0-baa8-ce843f6840bb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:16.321923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc5373-40c4-4f39-9d88-a3081208aa04 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:16.324037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 132/256] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:16 2022] GET /identity/v3/role_assignments?user.id=fca02058f3cd469688130fca81d450bd&scope.project.id=77149f8497f3433c8a1c236067c9eb62&role.id=6d1a0644f0bf45b4aa0f9fd0b3843261 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:16.324706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879aeb24-b688-44d0-baa8-ce843f6840bb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:16.368943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945de1d8-176f-40cd-bd82-5b1e8909b5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:16.369662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945de1d8-176f-40cd-bd82-5b1e8909b5cb None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:16.369946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945de1d8-176f-40cd-bd82-5b1e8909b5cb None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:16.370221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945de1d8-176f-40cd-bd82-5b1e8909b5cb None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:16.371073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945de1d8-176f-40cd-bd82-5b1e8909b5cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:16.371406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945de1d8-176f-40cd-bd82-5b1e8909b5cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:16.372900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945de1d8-176f-40cd-bd82-5b1e8909b5cb None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:16.375443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:16.386703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 125/257] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:16.392221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697a045-7b68-4867-b82b-72506caa02fa None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:16.392221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697a045-7b68-4867-b82b-72506caa02fa None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:16.392221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697a045-7b68-4867-b82b-72506caa02fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:16.408573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-0697a045-7b68-4867-b82b-72506caa02fa None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:16.409962 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0697a045-7b68-4867-b82b-72506caa02fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:16.503378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 126/258] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:16.504064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697a045-7b68-4867-b82b-72506caa02fa None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:16.512974 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879aeb24-b688-44d0-baa8-ce843f6840bb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:16.572567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f94a6-1d46-4488-b5e7-d1bc667a521e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:16.573471 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f94a6-1d46-4488-b5e7-d1bc667a521e None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:16.573754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f94a6-1d46-4488-b5e7-d1bc667a521e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:16.574028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f94a6-1d46-4488-b5e7-d1bc667a521e None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62/users/227f36fe6ea8435ca96db65897793e10/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:16.576069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f94a6-1d46-4488-b5e7-d1bc667a521e None admin] RBAC: Authorizing `identity:create_grant(project_id=77149f8497f3433c8a1c236067c9eb62, user_id=227f36fe6ea8435ca96db65897793e10, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:16.577547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f94a6-1d46-4488-b5e7-d1bc667a521e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:16.622691 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697a045-7b68-4867-b82b-72506caa02fa None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:16.635272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 133/259] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:39:16 2022] PUT /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62/users/227f36fe6ea8435ca96db65897793e10/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:39:16.640995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f94a6-1d46-4488-b5e7-d1bc667a521e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:16.655925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:16.675160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67b51f-7320-44f0-a63e-e14ad6210f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:16.675822 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:16.676115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67b51f-7320-44f0-a63e-e14ad6210f8a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:16.676737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67b51f-7320-44f0-a63e-e14ad6210f8a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:16.678923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67b51f-7320-44f0-a63e-e14ad6210f8a None admin] PATH_INFO: `/v3/users/7a40fca8538d4e87883ceda5e4d57c33` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:16.679366 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67b51f-7320-44f0-a63e-e14ad6210f8a None admin] RBAC: Authorizing `identity:get_user(user_id=7a40fca8538d4e87883ceda5e4d57c33)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:16.680844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67b51f-7320-44f0-a63e-e14ad6210f8a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:16.682659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 127/260] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:16 2022] GET /identity/v3/users/7a40fca8538d4e87883ceda5e4d57c33 => generated 284 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:16.688495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67b51f-7320-44f0-a63e-e14ad6210f8a None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:16.858799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f94a6-1d46-4488-b5e7-d1bc667a521e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:16.868572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67b51f-7320-44f0-a63e-e14ad6210f8a None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:16.925853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4ea37-68d8-441b-a7c3-e8e6b8e20f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:16.928131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4ea37-68d8-441b-a7c3-e8e6b8e20f76 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:16.928414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4ea37-68d8-441b-a7c3-e8e6b8e20f76 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:16.928783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4ea37-68d8-441b-a7c3-e8e6b8e20f76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:16.939906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a829ffb-b809-45c1-a260-c57ee1df40ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:16.940573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829ffb-b809-45c1-a260-c57ee1df40ca None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:16.940854 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829ffb-b809-45c1-a260-c57ee1df40ca None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:16.941130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829ffb-b809-45c1-a260-c57ee1df40ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:16.951686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4ea37-68d8-441b-a7c3-e8e6b8e20f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-91a4ea37-68d8-441b-a7c3-e8e6b8e20f76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:16.959933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:16.961011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:16.961874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:16.961874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:16.961874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:16.961874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:16.961874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:16.961874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:16.961874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:16.961874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:16.961874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:16.961874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:16.961874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:16.961874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:16.966836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 134/261] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:39:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:16.973533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4ea37-68d8-441b-a7c3-e8e6b8e20f76 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:17.091421 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:17.096592 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:17.108086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4ea37-68d8-441b-a7c3-e8e6b8e20f76 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:17.123678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a829ffb-b809-45c1-a260-c57ee1df40ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:17.126961 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a829ffb-b809-45c1-a260-c57ee1df40ca None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:17.144273 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e06a8d-a29c-4a80-abf0-09b8dd73f81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:17.145676 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e06a8d-a29c-4a80-abf0-09b8dd73f81d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:17.146135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e06a8d-a29c-4a80-abf0-09b8dd73f81d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:17.146578 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e06a8d-a29c-4a80-abf0-09b8dd73f81d None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:17.147558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e06a8d-a29c-4a80-abf0-09b8dd73f81d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:17.148084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e06a8d-a29c-4a80-abf0-09b8dd73f81d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:17.150287 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e06a8d-a29c-4a80-abf0-09b8dd73f81d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:17.159434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 135/262] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:16 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:17.165537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e06a8d-a29c-4a80-abf0-09b8dd73f81d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:17.331753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e06a8d-a29c-4a80-abf0-09b8dd73f81d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:17.336130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 128/263] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:16 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:39:17.341815 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a829ffb-b809-45c1-a260-c57ee1df40ca None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:17.370467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fc6f7-b44c-428d-9e6c-f6e49dbf1be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:17.371760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fc6f7-b44c-428d-9e6c-f6e49dbf1be4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:17.372259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fc6f7-b44c-428d-9e6c-f6e49dbf1be4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:17.372724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fc6f7-b44c-428d-9e6c-f6e49dbf1be4 None admin] PATH_INFO: `/v3/users/7a40fca8538d4e87883ceda5e4d57c33` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:17.377437 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fc6f7-b44c-428d-9e6c-f6e49dbf1be4 None admin] RBAC: Authorizing `identity:get_user(user_id=7a40fca8538d4e87883ceda5e4d57c33)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:17.379160 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fc6f7-b44c-428d-9e6c-f6e49dbf1be4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:17.382719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 136/264] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:17 2022] GET /identity/v3/users/7a40fca8538d4e87883ceda5e4d57c33 => generated 284 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:17.388635 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fc6f7-b44c-428d-9e6c-f6e49dbf1be4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:17.407302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:17.536289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:17.554956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a829ffb-b809-45c1-a260-c57ee1df40ca None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:17.606837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fc6f7-b44c-428d-9e6c-f6e49dbf1be4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:17.625940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fcfa0-3593-4127-a308-8d8359a0c913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:17.627516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fcfa0-3593-4127-a308-8d8359a0c913 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:17.627996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fcfa0-3593-4127-a308-8d8359a0c913 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:17.628463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fcfa0-3593-4127-a308-8d8359a0c913 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:17.632120 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fcfa0-3593-4127-a308-8d8359a0c913 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:17.633865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fcfa0-3593-4127-a308-8d8359a0c913 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:17.640042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 129/265] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:17 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:17.644405 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fcfa0-3593-4127-a308-8d8359a0c913 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:17.683068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3d903-d624-4dd1-a46b-eceb9a3aee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:17.684093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3d903-d624-4dd1-a46b-eceb9a3aee21 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:17.684569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3d903-d624-4dd1-a46b-eceb9a3aee21 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:17.685062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3d903-d624-4dd1-a46b-eceb9a3aee21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:17.828650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3d903-d624-4dd1-a46b-eceb9a3aee21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:17.831122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3d903-d624-4dd1-a46b-eceb9a3aee21 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:17.845615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:17.847996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fcfa0-3593-4127-a308-8d8359a0c913 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:17.932725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757ee00-16d0-4fd8-8ca8-463ead509982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:17.933340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757ee00-16d0-4fd8-8ca8-463ead509982 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:17.933622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757ee00-16d0-4fd8-8ca8-463ead509982 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:17.934062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757ee00-16d0-4fd8-8ca8-463ead509982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:17.934948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757ee00-16d0-4fd8-8ca8-463ead509982 None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=77149f8497f3433c8a1c236067c9eb62, user.id=7a40fca8538d4e87883ceda5e4d57c33) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:17.935444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757ee00-16d0-4fd8-8ca8-463ead509982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:17.937093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757ee00-16d0-4fd8-8ca8-463ead509982 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:17.949554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 130/266] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:17 2022] GET /identity/v3/role_assignments?user.id=7a40fca8538d4e87883ceda5e4d57c33&scope.project.id=77149f8497f3433c8a1c236067c9eb62&role.id=4f574961798246a4b12ad2276e67685d => generated 610 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:17.986078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 137/267] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:17 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:17.992057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757ee00-16d0-4fd8-8ca8-463ead509982 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:18.041121 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:18.137899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e7cd2-2381-4a90-add2-c377610204f9 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:18.138520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e7cd2-2381-4a90-add2-c377610204f9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:18.139125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e7cd2-2381-4a90-add2-c377610204f9 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:18.140759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 138/268] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:18.147557 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6b786-78bc-4735-993b-9bd16267253c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:18.148597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6b786-78bc-4735-993b-9bd16267253c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:18.149070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6b786-78bc-4735-993b-9bd16267253c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:18.182934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757ee00-16d0-4fd8-8ca8-463ead509982 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:18.184943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e4b6b786-78bc-4735-993b-9bd16267253c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:18.186606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e4b6b786-78bc-4735-993b-9bd16267253c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:18.263746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50495651-2dd2-4b09-b03c-6ce9dede0560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:18.265036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50495651-2dd2-4b09-b03c-6ce9dede0560 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:18.265324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50495651-2dd2-4b09-b03c-6ce9dede0560 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:18.265607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50495651-2dd2-4b09-b03c-6ce9dede0560 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:18.268727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50495651-2dd2-4b09-b03c-6ce9dede0560 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:18.270328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50495651-2dd2-4b09-b03c-6ce9dede0560 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:18.273070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 131/269] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:17 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:18.279830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50495651-2dd2-4b09-b03c-6ce9dede0560 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:18.330533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 139/270] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:18.386854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6b786-78bc-4735-993b-9bd16267253c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:18.413611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:18.422846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50495651-2dd2-4b09-b03c-6ce9dede0560 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:18.460786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705312a4-cd56-4266-8d9f-0b929728f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:18.461510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705312a4-cd56-4266-8d9f-0b929728f5bb None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:18.461799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705312a4-cd56-4266-8d9f-0b929728f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:18.462110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705312a4-cd56-4266-8d9f-0b929728f5bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:18.462968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705312a4-cd56-4266-8d9f-0b929728f5bb None admin] RBAC: Adding query filter params (role.id=582f59fc05ce474287c6fdc503f9ac82, scope.project.id=77149f8497f3433c8a1c236067c9eb62, user.id=7a40fca8538d4e87883ceda5e4d57c33) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:18.463285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705312a4-cd56-4266-8d9f-0b929728f5bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:18.464996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705312a4-cd56-4266-8d9f-0b929728f5bb None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:18.474343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 132/271] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:18 2022] GET /identity/v3/role_assignments?user.id=7a40fca8538d4e87883ceda5e4d57c33&scope.project.id=77149f8497f3433c8a1c236067c9eb62&role.id=582f59fc05ce474287c6fdc503f9ac82 => generated 610 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:18.503008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:18.600251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6b786-78bc-4735-993b-9bd16267253c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:18.615922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f6aa3-e79a-4c81-9efb-b8c807f410a5 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:18.616302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f6aa3-e79a-4c81-9efb-b8c807f410a5 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:18.616475 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f6aa3-e79a-4c81-9efb-b8c807f410a5 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:18.618121 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 133/272] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:18.624163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf07a9-f6d3-4a57-b6e1-0c48b1836b48 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:18.624441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf07a9-f6d3-4a57-b6e1-0c48b1836b48 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:18.624741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf07a9-f6d3-4a57-b6e1-0c48b1836b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:18.653996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472bb32-7d89-4da9-b21f-b3d58d92ffe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:18.654805 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472bb32-7d89-4da9-b21f-b3d58d92ffe3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:18.655109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472bb32-7d89-4da9-b21f-b3d58d92ffe3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:18.655381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472bb32-7d89-4da9-b21f-b3d58d92ffe3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:18.656576 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472bb32-7d89-4da9-b21f-b3d58d92ffe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:18.658059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472bb32-7d89-4da9-b21f-b3d58d92ffe3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:18.660111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 140/273] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:18 2022] GET /identity/v3/domains/default => generated 200 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:18.661627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-2cbf07a9-f6d3-4a57-b6e1-0c48b1836b48 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:18.663039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-2cbf07a9-f6d3-4a57-b6e1-0c48b1836b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:18.666156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472bb32-7d89-4da9-b21f-b3d58d92ffe3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:18.789683 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 134/274] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:18.818790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf07a9-f6d3-4a57-b6e1-0c48b1836b48 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:18.837914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472bb32-7d89-4da9-b21f-b3d58d92ffe3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:18.840759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:18.877357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b06025-08dd-4e6b-8a50-88d68298c247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:18.879003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b06025-08dd-4e6b-8a50-88d68298c247 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:18.880288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b06025-08dd-4e6b-8a50-88d68298c247 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:18.881288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b06025-08dd-4e6b-8a50-88d68298c247 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:18.882916 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b06025-08dd-4e6b-8a50-88d68298c247 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:18.884643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b06025-08dd-4e6b-8a50-88d68298c247 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:18.928359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:18.944566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf07a9-f6d3-4a57-b6e1-0c48b1836b48 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:18.990805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e3012-b053-4818-8395-f15460853ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:18.991635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e3012-b053-4818-8395-f15460853ba2 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:18.991917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e3012-b053-4818-8395-f15460853ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:18.992188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e3012-b053-4818-8395-f15460853ba2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:19.005681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e3012-b053-4818-8395-f15460853ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:19.011220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 141/275] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:39:18 2022] POST /identity/v3/users => generated 294 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-b24e3012-b053-4818-8395-f15460853ba2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:19.015231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:19.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:19.017021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 135/276] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:19.022219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b06025-08dd-4e6b-8a50-88d68298c247 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:19.153520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b06025-08dd-4e6b-8a50-88d68298c247 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:19.197535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc128c9-cf3c-4a59-8ccb-87e285e6a89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:19.198356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc128c9-cf3c-4a59-8ccb-87e285e6a89e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:19.198728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc128c9-cf3c-4a59-8ccb-87e285e6a89e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:19.199054 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc128c9-cf3c-4a59-8ccb-87e285e6a89e None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:19.199851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc128c9-cf3c-4a59-8ccb-87e285e6a89e None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:19.200160 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc128c9-cf3c-4a59-8ccb-87e285e6a89e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:19.201599 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc128c9-cf3c-4a59-8ccb-87e285e6a89e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:19.213241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 142/277] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:19 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:19.218949 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e3012-b053-4818-8395-f15460853ba2 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:19.338481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e3012-b053-4818-8395-f15460853ba2 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:19.374599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c7fee-e178-4181-8f64-16fb3f104348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:19.375511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c7fee-e178-4181-8f64-16fb3f104348 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:19.375788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c7fee-e178-4181-8f64-16fb3f104348 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:19.376057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c7fee-e178-4181-8f64-16fb3f104348 None admin] PATH_INFO: `/v3/users/dc4b60c90c48454383c96a4977c507e8` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:19.391692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c7fee-e178-4181-8f64-16fb3f104348 None admin] RBAC: Authorizing `identity:get_user(user_id=dc4b60c90c48454383c96a4977c507e8)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:19.393183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c7fee-e178-4181-8f64-16fb3f104348 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:19.395102 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 136/278] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:19 2022] GET /identity/v3/users/dc4b60c90c48454383c96a4977c507e8 => generated 294 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:19.401351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc128c9-cf3c-4a59-8ccb-87e285e6a89e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:19.421803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c6f92-aae3-4e97-ade5-8b898def2473 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:19.422308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c6f92-aae3-4e97-ade5-8b898def2473 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:19.422380 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c6f92-aae3-4e97-ade5-8b898def2473 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:19.424010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 137/279] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:39:19.430634 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926246f-ff60-443e-ab52-4b31e86e3d7b None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:19.430948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926246f-ff60-443e-ab52-4b31e86e3d7b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:19.431229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926246f-ff60-443e-ab52-4b31e86e3d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:19.433097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:19.445156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:19.456396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1926246f-ff60-443e-ab52-4b31e86e3d7b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:19.458031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1926246f-ff60-443e-ab52-4b31e86e3d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:19.532652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc128c9-cf3c-4a59-8ccb-87e285e6a89e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:19.535016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 138/280] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:19.559858 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926246f-ff60-443e-ab52-4b31e86e3d7b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:19.583963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474992bb-d851-4531-9f90-4e24bbb05a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:19.585256 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474992bb-d851-4531-9f90-4e24bbb05a71 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:19.585825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474992bb-d851-4531-9f90-4e24bbb05a71 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:19.586299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474992bb-d851-4531-9f90-4e24bbb05a71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:19.587935 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474992bb-d851-4531-9f90-4e24bbb05a71 None admin] RBAC: Adding query filter params (role.id=e87c07d8d70d46d89a2c4f7246edeff6, user.id=dc4b60c90c48454383c96a4977c507e8, scope.system=all) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:19.588463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474992bb-d851-4531-9f90-4e24bbb05a71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:19.590233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474992bb-d851-4531-9f90-4e24bbb05a71 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:19.596312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 143/281] 10.222.0.121 () {56 vars in 1313 bytes} [Thu May 26 17:39:19 2022] GET /identity/v3/role_assignments?user.id=dc4b60c90c48454383c96a4977c507e8&scope.system=all&role.id=e87c07d8d70d46d89a2c4f7246edeff6 => generated 227 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:19.679250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926246f-ff60-443e-ab52-4b31e86e3d7b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:19.719649 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bf53c-ee05-4534-b514-96e754b30cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:19.720488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bf53c-ee05-4534-b514-96e754b30cac None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:19.721740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bf53c-ee05-4534-b514-96e754b30cac None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:19.721740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bf53c-ee05-4534-b514-96e754b30cac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:19.727900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:19.735217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bf53c-ee05-4534-b514-96e754b30cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d56bf53c-ee05-4534-b514-96e754b30cac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:19.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:19.745529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:19.746330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 139/282] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:19.753735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474992bb-d851-4531-9f90-4e24bbb05a71 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:19.905265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:19.921310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474992bb-d851-4531-9f90-4e24bbb05a71 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:19.944748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf56822-af36-4979-b408-098077a91f95 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:19.945028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf56822-af36-4979-b408-098077a91f95 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:19.945205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf56822-af36-4979-b408-098077a91f95 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:19.946693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 140/283] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:19 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:19.952191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1211468-1a45-4b4c-83ab-dc97751b4d57 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:19.952459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1211468-1a45-4b4c-83ab-dc97751b4d57 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:19.952736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1211468-1a45-4b4c-83ab-dc97751b4d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:19.981982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56ca5b-fe4f-4e86-a418-a8e4e8faeb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:19.982657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56ca5b-fe4f-4e86-a418-a8e4e8faeb16 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:19.982969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56ca5b-fe4f-4e86-a418-a8e4e8faeb16 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:19.983298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56ca5b-fe4f-4e86-a418-a8e4e8faeb16 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:19.983665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b1211468-1a45-4b4c-83ab-dc97751b4d57 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:19.984053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56ca5b-fe4f-4e86-a418-a8e4e8faeb16 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:19.984359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56ca5b-fe4f-4e86-a418-a8e4e8faeb16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:19.985567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b1211468-1a45-4b4c-83ab-dc97751b4d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:19.985766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56ca5b-fe4f-4e86-a418-a8e4e8faeb16 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:19.995637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 144/284] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:19 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:20.001271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56ca5b-fe4f-4e86-a418-a8e4e8faeb16 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:20.122139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 141/285] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:20.175766 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1211468-1a45-4b4c-83ab-dc97751b4d57 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:20.189106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56ca5b-fe4f-4e86-a418-a8e4e8faeb16 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:20.253625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40850547-7a54-4637-affc-e595a44db0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:20.254427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40850547-7a54-4637-affc-e595a44db0a8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:20.254749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40850547-7a54-4637-affc-e595a44db0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:20.255055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40850547-7a54-4637-affc-e595a44db0a8 None admin] PATH_INFO: `/v3/users/fca02058f3cd469688130fca81d450bd` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:20.256277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40850547-7a54-4637-affc-e595a44db0a8 None admin] RBAC: Authorizing `identity:get_user(user_id=fca02058f3cd469688130fca81d450bd)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:20.257735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40850547-7a54-4637-affc-e595a44db0a8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:20.261483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 145/286] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:20 2022] GET /identity/v3/users/fca02058f3cd469688130fca81d450bd => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:20.266782 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40850547-7a54-4637-affc-e595a44db0a8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:20.292166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:20.360137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1211468-1a45-4b4c-83ab-dc97751b4d57 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:20.410189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:20.427867 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc6d31-cb9e-41da-a1f0-fd8bb9408c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:20.428680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc6d31-cb9e-41da-a1f0-fd8bb9408c1b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:20.429220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc6d31-cb9e-41da-a1f0-fd8bb9408c1b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:20.429494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc6d31-cb9e-41da-a1f0-fd8bb9408c1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:20.441294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40850547-7a54-4637-affc-e595a44db0a8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:20.452732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc6d31-cb9e-41da-a1f0-fd8bb9408c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-fbcc6d31-cb9e-41da-a1f0-fd8bb9408c1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:20.464569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:20.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:20.466281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 142/287] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:20.475429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc6d31-cb9e-41da-a1f0-fd8bb9408c1b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:20.509725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b104b-4355-417c-880b-5b6e663a4be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:20.510995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b104b-4355-417c-880b-5b6e663a4be4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:20.511279 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b104b-4355-417c-880b-5b6e663a4be4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:20.511556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b104b-4355-417c-880b-5b6e663a4be4 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:20.512778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b104b-4355-417c-880b-5b6e663a4be4 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:20.514306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b104b-4355-417c-880b-5b6e663a4be4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:20.516848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 146/288] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:20 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:20.522497 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b104b-4355-417c-880b-5b6e663a4be4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:20.673835 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc6d31-cb9e-41da-a1f0-fd8bb9408c1b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:20.679254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:20.688949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b104b-4355-417c-880b-5b6e663a4be4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:20.743589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734fe11-d1ee-4e07-ba16-a95ab88d60a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:20.744509 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734fe11-d1ee-4e07-ba16-a95ab88d60a3 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:20.744788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734fe11-d1ee-4e07-ba16-a95ab88d60a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:20.745063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734fe11-d1ee-4e07-ba16-a95ab88d60a3 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62/users/fca02058f3cd469688130fca81d450bd/roles/6d1a0644f0bf45b4aa0f9fd0b3843261` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:20.749138 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734fe11-d1ee-4e07-ba16-a95ab88d60a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=77149f8497f3433c8a1c236067c9eb62, user_id=fca02058f3cd469688130fca81d450bd, role_id=6d1a0644f0bf45b4aa0f9fd0b3843261)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:20.750605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734fe11-d1ee-4e07-ba16-a95ab88d60a3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:20.755172 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddd4c9-aa7b-498f-8681-890a67c8fcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:20.756064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddd4c9-aa7b-498f-8681-890a67c8fcbe None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:20.756352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddd4c9-aa7b-498f-8681-890a67c8fcbe None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:20.756630 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddd4c9-aa7b-498f-8681-890a67c8fcbe None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:20.757390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaddd4c9-aa7b-498f-8681-890a67c8fcbe None admin] RBAC: Adding query filter params (name=member) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:20.757705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaddd4c9-aa7b-498f-8681-890a67c8fcbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:20.759158 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaddd4c9-aa7b-498f-8681-890a67c8fcbe None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:20.770998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 143/289] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:20 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:20.776268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddd4c9-aa7b-498f-8681-890a67c8fcbe None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:20.845977 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 147/290] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:39:20 2022] PUT /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62/users/fca02058f3cd469688130fca81d450bd/roles/6d1a0644f0bf45b4aa0f9fd0b3843261 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:39:20.847927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:20.907147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddd4c9-aa7b-498f-8681-890a67c8fcbe None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:20.956344 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decf869-aac4-4224-976c-e97ff24952d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:20.957232 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decf869-aac4-4224-976c-e97ff24952d6 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:20.957537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decf869-aac4-4224-976c-e97ff24952d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:20.957865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decf869-aac4-4224-976c-e97ff24952d6 None admin] PATH_INFO: `/v3/users/227f36fe6ea8435ca96db65897793e10` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:20.963648 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7decf869-aac4-4224-976c-e97ff24952d6 None admin] RBAC: Authorizing `identity:get_user(user_id=227f36fe6ea8435ca96db65897793e10)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:20.965152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7decf869-aac4-4224-976c-e97ff24952d6 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:20.967417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 144/291] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:20 2022] GET /identity/v3/users/227f36fe6ea8435ca96db65897793e10 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:20.972596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734fe11-d1ee-4e07-ba16-a95ab88d60a3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:21.111735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734fe11-d1ee-4e07-ba16-a95ab88d60a3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:21.165759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0917b-a2bb-49ca-84c1-4323af2e33c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:21.166452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0917b-a2bb-49ca-84c1-4323af2e33c0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:21.166735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0917b-a2bb-49ca-84c1-4323af2e33c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:21.167447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0917b-a2bb-49ca-84c1-4323af2e33c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:21.189044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:21.263736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0917b-a2bb-49ca-84c1-4323af2e33c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:21.265196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0917b-a2bb-49ca-84c1-4323af2e33c0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:21.373122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 148/292] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:20 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:21.378633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decf869-aac4-4224-976c-e97ff24952d6 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:21.497607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decf869-aac4-4224-976c-e97ff24952d6 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:21.523191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:21.551628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99255c7e-9948-4a45-aa14-4c0343117056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:21.552670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99255c7e-9948-4a45-aa14-4c0343117056 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:21.552956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99255c7e-9948-4a45-aa14-4c0343117056 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:21.553229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99255c7e-9948-4a45-aa14-4c0343117056 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:21.554680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99255c7e-9948-4a45-aa14-4c0343117056 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:21.556149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99255c7e-9948-4a45-aa14-4c0343117056 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:21.558169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 145/293] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:21 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:21.568432 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0917b-a2bb-49ca-84c1-4323af2e33c0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:21.656087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:21.707008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0917b-a2bb-49ca-84c1-4323af2e33c0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:21.744464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07bb90-3af8-4200-8f6d-a3cf10988c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:21.745131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07bb90-3af8-4200-8f6d-a3cf10988c51 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:21.745416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07bb90-3af8-4200-8f6d-a3cf10988c51 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:21.745701 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07bb90-3af8-4200-8f6d-a3cf10988c51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:21.746519 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07bb90-3af8-4200-8f6d-a3cf10988c51 None admin] RBAC: Adding query filter params (role.id=e87c07d8d70d46d89a2c4f7246edeff6, scope.project.id=77149f8497f3433c8a1c236067c9eb62, user.id=227f36fe6ea8435ca96db65897793e10) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:21.746843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07bb90-3af8-4200-8f6d-a3cf10988c51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:21.748381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07bb90-3af8-4200-8f6d-a3cf10988c51 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:21.755462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 149/294] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:21 2022] GET /identity/v3/role_assignments?user.id=227f36fe6ea8435ca96db65897793e10&scope.project.id=77149f8497f3433c8a1c236067c9eb62&role.id=e87c07d8d70d46d89a2c4f7246edeff6 => generated 610 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:21.856918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c73c7-21f9-4e58-bb28-2fb636570398 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:21.857269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c73c7-21f9-4e58-bb28-2fb636570398 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:21.857469 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c73c7-21f9-4e58-bb28-2fb636570398 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:21.858865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 146/295] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:21.873454 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596e139-5e7c-4f13-842a-f7532e12ff22 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:21.873827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596e139-5e7c-4f13-842a-f7532e12ff22 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:21.874010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596e139-5e7c-4f13-842a-f7532e12ff22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:21.897733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9596e139-5e7c-4f13-842a-f7532e12ff22 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:21.898983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9596e139-5e7c-4f13-842a-f7532e12ff22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:21.984087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 150/296] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:21 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:22.019585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c73c7-21f9-4e58-bb28-2fb636570398 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:22.072657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd1f4-2a84-44b0-bc4b-cd695b82e4ad None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:22.072938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd1f4-2a84-44b0-bc4b-cd695b82e4ad None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:22.073216 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd1f4-2a84-44b0-bc4b-cd695b82e4ad None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:22.074944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 151/297] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:22.079858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1b1c5-7b13-4eef-bf9c-0739963ea0a8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:22.080259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1b1c5-7b13-4eef-bf9c-0739963ea0a8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:22.080741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1b1c5-7b13-4eef-bf9c-0739963ea0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:22.102502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7ae1b1c5-7b13-4eef-bf9c-0739963ea0a8 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:22.105697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7ae1b1c5-7b13-4eef-bf9c-0739963ea0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:22.146804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:22.187896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c73c7-21f9-4e58-bb28-2fb636570398 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:22.203583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 152/298] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:22.234811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1b1c5-7b13-4eef-bf9c-0739963ea0a8 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:22.238222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e5a08-e391-486b-9605-7d5cc076a5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:22.239705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e5a08-e391-486b-9605-7d5cc076a5b3 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:22.240182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e5a08-e391-486b-9605-7d5cc076a5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:22.240598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e5a08-e391-486b-9605-7d5cc076a5b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:22.252549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e5a08-e391-486b-9605-7d5cc076a5b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-eb3e5a08-e391-486b-9605-7d5cc076a5b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:22.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:22.265043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:22.265826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:22.265826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:22.265826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:22.265826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:22.265826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:22.265826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:22.265826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:22.265826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:22.265826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:22.265826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:22.265826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:22.265826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:22.268138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 147/299] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:22.274127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e5a08-e391-486b-9605-7d5cc076a5b3 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:22.324178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:22.368221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1b1c5-7b13-4eef-bf9c-0739963ea0a8 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:22.383762 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e5a08-e391-486b-9605-7d5cc076a5b3 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:22.410117 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44c2e0-35a7-4f46-9301-33afcfdca0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:22.412363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44c2e0-35a7-4f46-9301-33afcfdca0d0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:22.412913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44c2e0-35a7-4f46-9301-33afcfdca0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:22.413334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44c2e0-35a7-4f46-9301-33afcfdca0d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:22.414854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44c2e0-35a7-4f46-9301-33afcfdca0d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:22.416701 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44c2e0-35a7-4f46-9301-33afcfdca0d0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:22.419345 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 153/300] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:22 2022] GET /identity/v3/domains/default => generated 200 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:22.424897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44c2e0-35a7-4f46-9301-33afcfdca0d0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:22.439517 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e79d4-9b56-4cda-a706-294bd45d07a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:22.441416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e79d4-9b56-4cda-a706-294bd45d07a9 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:22.441923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e79d4-9b56-4cda-a706-294bd45d07a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:22.442410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e79d4-9b56-4cda-a706-294bd45d07a9 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:22.443764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e79d4-9b56-4cda-a706-294bd45d07a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:22.444862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e79d4-9b56-4cda-a706-294bd45d07a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:22.446855 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e79d4-9b56-4cda-a706-294bd45d07a9 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:22.463922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 148/301] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:22 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:22.465341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e79d4-9b56-4cda-a706-294bd45d07a9 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:22.475868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:22.548824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44c2e0-35a7-4f46-9301-33afcfdca0d0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:22.576108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e79d4-9b56-4cda-a706-294bd45d07a9 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:22.584379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:22.588411 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dc191-ae46-47c8-a937-82b68501e59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:22.589101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dc191-ae46-47c8-a937-82b68501e59a None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:22.589384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dc191-ae46-47c8-a937-82b68501e59a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:22.589663 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dc191-ae46-47c8-a937-82b68501e59a None admin] PATH_INFO: `/v3/groups` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:22.590622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dc191-ae46-47c8-a937-82b68501e59a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:22.592005 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dc191-ae46-47c8-a937-82b68501e59a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:22.615778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfbd9a-4cbd-4a02-b7eb-81045676bbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:22.616763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfbd9a-4cbd-4a02-b7eb-81045676bbbc None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:22.617113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfbd9a-4cbd-4a02-b7eb-81045676bbbc None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:22.617479 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfbd9a-4cbd-4a02-b7eb-81045676bbbc None admin] PATH_INFO: `/v3/users/d1106a40711140aeba0f31c32c2c8edd` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:22.633260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfbd9a-4cbd-4a02-b7eb-81045676bbbc None admin] RBAC: Authorizing `identity:get_user(user_id=d1106a40711140aeba0f31c32c2c8edd)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:22.634729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfbd9a-4cbd-4a02-b7eb-81045676bbbc None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:22.636449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 149/302] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:22 2022] GET /identity/v3/users/d1106a40711140aeba0f31c32c2c8edd => generated 294 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:22.638764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7799c6-b882-4839-95ed-7454d966a94d None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:22.639161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7799c6-b882-4839-95ed-7454d966a94d None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:22.639563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7799c6-b882-4839-95ed-7454d966a94d None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:22.640732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 150/303] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:22.641984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7799c6-b882-4839-95ed-7454d966a94d None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:22.695013 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 154/304] 10.222.0.121 () {60 vars in 1139 bytes} [Thu May 26 17:39:22 2022] POST /identity/v3/groups => generated 230 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:39:22.696854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3a30c-9e5c-4be5-bc3b-02f0931b5819 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:22.697361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3a30c-9e5c-4be5-bc3b-02f0931b5819 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:22.697830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3a30c-9e5c-4be5-bc3b-02f0931b5819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:22.727767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-72a3a30c-9e5c-4be5-bc3b-02f0931b5819 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:22.730395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-72a3a30c-9e5c-4be5-bc3b-02f0931b5819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:22.773176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7799c6-b882-4839-95ed-7454d966a94d None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:22.822035 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 155/305] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:22.832318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ff81d-c066-4b5a-8e88-ed72f4fc0046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:22.834087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ff81d-c066-4b5a-8e88-ed72f4fc0046 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:22.835291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ff81d-c066-4b5a-8e88-ed72f4fc0046 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:22.836108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ff81d-c066-4b5a-8e88-ed72f4fc0046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:22.838961 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ff81d-c066-4b5a-8e88-ed72f4fc0046 None admin] RBAC: Adding query filter params (role.id=6d1a0644f0bf45b4aa0f9fd0b3843261, user.id=d1106a40711140aeba0f31c32c2c8edd, scope.system=all) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:22.839545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ff81d-c066-4b5a-8e88-ed72f4fc0046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:22.841388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ff81d-c066-4b5a-8e88-ed72f4fc0046 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:22.846795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3a30c-9e5c-4be5-bc3b-02f0931b5819 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:22.854861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 151/306] 10.222.0.121 () {56 vars in 1313 bytes} [Thu May 26 17:39:22 2022] GET /identity/v3/role_assignments?user.id=d1106a40711140aeba0f31c32c2c8edd&scope.system=all&role.id=6d1a0644f0bf45b4aa0f9fd0b3843261 => generated 227 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:22.888212 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:22.986143 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3a30c-9e5c-4be5-bc3b-02f0931b5819 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:23.031789 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d8059-afc7-47a7-970e-83038b45a40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:23.032637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d8059-afc7-47a7-970e-83038b45a40a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:23.032972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d8059-afc7-47a7-970e-83038b45a40a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:23.033253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d8059-afc7-47a7-970e-83038b45a40a None admin] PATH_INFO: `/v3/roles/member` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:23.048435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d8059-afc7-47a7-970e-83038b45a40a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e90d8059-afc7-47a7-970e-83038b45a40a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:23.056371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:23.057414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:23.058200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:23.058200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:23.058200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:23.058200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:23.058200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:23.058200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:23.058200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:23.058200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:23.058200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:23.058200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:23.058200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:23.058200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:23.062315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 156/307] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:23.069405 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ff81d-c066-4b5a-8e88-ed72f4fc0046 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:23.105579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:23.200368 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ff81d-c066-4b5a-8e88-ed72f4fc0046 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:23.242579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a58eb-7692-49dc-9685-798dc9f85550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:23.243885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a58eb-7692-49dc-9685-798dc9f85550 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:23.244372 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a58eb-7692-49dc-9685-798dc9f85550 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:23.244838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a58eb-7692-49dc-9685-798dc9f85550 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:23.245864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a58eb-7692-49dc-9685-798dc9f85550 None admin] RBAC: Adding query filter params (name=member) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:23.246361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a58eb-7692-49dc-9685-798dc9f85550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:23.248056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a58eb-7692-49dc-9685-798dc9f85550 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:23.258957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 152/308] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:23 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:23.264615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d8059-afc7-47a7-970e-83038b45a40a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:23.366039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:23.383535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d8059-afc7-47a7-970e-83038b45a40a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:23.422034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e644a7-71ff-4873-8d61-a6d47e4b7c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:23.423258 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e644a7-71ff-4873-8d61-a6d47e4b7c5c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:23.423721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e644a7-71ff-4873-8d61-a6d47e4b7c5c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:23.424174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e644a7-71ff-4873-8d61-a6d47e4b7c5c None admin] PATH_INFO: `/v3/users/dc4b60c90c48454383c96a4977c507e8` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:23.425583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e644a7-71ff-4873-8d61-a6d47e4b7c5c None admin] RBAC: Authorizing `identity:get_user(user_id=dc4b60c90c48454383c96a4977c507e8)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:23.427596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e644a7-71ff-4873-8d61-a6d47e4b7c5c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:23.434233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 157/309] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:23 2022] GET /identity/v3/users/dc4b60c90c48454383c96a4977c507e8 => generated 294 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:23.435953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a58eb-7692-49dc-9685-798dc9f85550 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:23.552601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:23.562283 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a58eb-7692-49dc-9685-798dc9f85550 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:23.605167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38fa37-bdf2-4e97-b306-28d39a0e9c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:23.606051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38fa37-bdf2-4e97-b306-28d39a0e9c26 None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:23.606333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38fa37-bdf2-4e97-b306-28d39a0e9c26 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:23.606607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38fa37-bdf2-4e97-b306-28d39a0e9c26 None admin] PATH_INFO: `/v3/system/users/dc4b60c90c48454383c96a4977c507e8/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:23.608153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38fa37-bdf2-4e97-b306-28d39a0e9c26 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dc4b60c90c48454383c96a4977c507e8, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:23.609636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38fa37-bdf2-4e97-b306-28d39a0e9c26 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:23.720440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9d2e9-6e49-4970-9a33-db31ed822c54 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:23.720739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9d2e9-6e49-4970-9a33-db31ed822c54 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:23.721186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9d2e9-6e49-4970-9a33-db31ed822c54 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:23.724774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 158/310] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:23 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:39:23.728036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 153/311] 10.222.0.121 () {58 vars in 1339 bytes} [Thu May 26 17:39:23 2022] PUT /identity/v3/system/users/dc4b60c90c48454383c96a4977c507e8/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:39:23.734856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657e415-d889-4a4f-9d0e-6baac451daf7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:23.734856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657e415-d889-4a4f-9d0e-6baac451daf7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:23.734856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657e415-d889-4a4f-9d0e-6baac451daf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:23.760291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2657e415-d889-4a4f-9d0e-6baac451daf7 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:23.761591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2657e415-d889-4a4f-9d0e-6baac451daf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:23.841506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 159/312] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:23.865647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38fa37-bdf2-4e97-b306-28d39a0e9c26 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:23.990210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38fa37-bdf2-4e97-b306-28d39a0e9c26 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:24.028381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f31d6-81df-414b-9ad9-7aa8d555f772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:24.029550 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f31d6-81df-414b-9ad9-7aa8d555f772 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:24.030034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f31d6-81df-414b-9ad9-7aa8d555f772 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:24.030504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f31d6-81df-414b-9ad9-7aa8d555f772 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:24.042764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f31d6-81df-414b-9ad9-7aa8d555f772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d52f31d6-81df-414b-9ad9-7aa8d555f772 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:24.050841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:24.052362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:24.053147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:24.053147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:24.053147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:24.053147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:24.053147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:24.053147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:24.053147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:24.053147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:24.053147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:24.053147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:24.053147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:24.053147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:24.055618 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 154/313] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:24.064227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657e415-d889-4a4f-9d0e-6baac451daf7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:24.160500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:24.183390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657e415-d889-4a4f-9d0e-6baac451daf7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:24.220959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a575b-dfd5-4908-8815-4cc83e2aacc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:24.221964 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a575b-dfd5-4908-8815-4cc83e2aacc0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:24.222388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a575b-dfd5-4908-8815-4cc83e2aacc0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:24.222938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a575b-dfd5-4908-8815-4cc83e2aacc0 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:24.224030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a575b-dfd5-4908-8815-4cc83e2aacc0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:24.224504 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a575b-dfd5-4908-8815-4cc83e2aacc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:24.226711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a575b-dfd5-4908-8815-4cc83e2aacc0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:24.236213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 160/314] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:24 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:24.242920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f31d6-81df-414b-9ad9-7aa8d555f772 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:24.259370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:24.351196 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f31d6-81df-414b-9ad9-7aa8d555f772 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:24.386829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8687b06-8b15-48c2-9e2a-fada714f5db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:24.387721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8687b06-8b15-48c2-9e2a-fada714f5db8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:24.388002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8687b06-8b15-48c2-9e2a-fada714f5db8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:24.388276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8687b06-8b15-48c2-9e2a-fada714f5db8 None admin] PATH_INFO: `/v3/users/fca02058f3cd469688130fca81d450bd` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:24.389487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8687b06-8b15-48c2-9e2a-fada714f5db8 None admin] RBAC: Authorizing `identity:get_user(user_id=fca02058f3cd469688130fca81d450bd)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:24.391035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8687b06-8b15-48c2-9e2a-fada714f5db8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:24.393049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 155/315] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:24 2022] GET /identity/v3/users/fca02058f3cd469688130fca81d450bd => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:24.399033 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a575b-dfd5-4908-8815-4cc83e2aacc0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:24.504242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a575b-dfd5-4908-8815-4cc83e2aacc0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:24.545529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e9e18-b043-4c38-94fe-a5879caa6db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:24.546196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e9e18-b043-4c38-94fe-a5879caa6db7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:24.546480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e9e18-b043-4c38-94fe-a5879caa6db7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:24.546760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e9e18-b043-4c38-94fe-a5879caa6db7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:24.563694 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:24.627278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e9e18-b043-4c38-94fe-a5879caa6db7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:24.628764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e9e18-b043-4c38-94fe-a5879caa6db7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:24.739131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 161/316] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:24 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:24.744930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8687b06-8b15-48c2-9e2a-fada714f5db8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:24.899406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:24.921343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8687b06-8b15-48c2-9e2a-fada714f5db8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:24.964157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f77d8-a38e-42e8-b550-d13b7b5d6503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:24.965173 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f77d8-a38e-42e8-b550-d13b7b5d6503 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:24.965498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f77d8-a38e-42e8-b550-d13b7b5d6503 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:24.965774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f77d8-a38e-42e8-b550-d13b7b5d6503 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:24.967408 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f77d8-a38e-42e8-b550-d13b7b5d6503 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:24.968939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f77d8-a38e-42e8-b550-d13b7b5d6503 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:24.970803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 156/317] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:24 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:24.977506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e9e18-b043-4c38-94fe-a5879caa6db7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:25.029787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:25.090791 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e9e18-b043-4c38-94fe-a5879caa6db7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:25.127932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dca31d-9c6f-4af7-8b21-c183857b7e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:25.128596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dca31d-9c6f-4af7-8b21-c183857b7e9e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:25.128875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dca31d-9c6f-4af7-8b21-c183857b7e9e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:25.129143 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dca31d-9c6f-4af7-8b21-c183857b7e9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:25.130207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dca31d-9c6f-4af7-8b21-c183857b7e9e None admin] RBAC: Adding query filter params (role.id=6d1a0644f0bf45b4aa0f9fd0b3843261, scope.project.id=77149f8497f3433c8a1c236067c9eb62, user.id=fca02058f3cd469688130fca81d450bd) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:25.130506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dca31d-9c6f-4af7-8b21-c183857b7e9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:25.131987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dca31d-9c6f-4af7-8b21-c183857b7e9e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:25.138437 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 162/318] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:24 2022] GET /identity/v3/role_assignments?user.id=fca02058f3cd469688130fca81d450bd&scope.project.id=77149f8497f3433c8a1c236067c9eb62&role.id=6d1a0644f0bf45b4aa0f9fd0b3843261 => generated 610 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:25.165291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f968fe-0556-4477-9667-62c98c3e7b20 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:25.165542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f968fe-0556-4477-9667-62c98c3e7b20 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:25.165762 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f968fe-0556-4477-9667-62c98c3e7b20 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:25.168012 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 157/319] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:25.175840 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f387727-44d3-4a71-97c8-c0d078bf4639 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:25.176253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f387727-44d3-4a71-97c8-c0d078bf4639 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:25.176592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f387727-44d3-4a71-97c8-c0d078bf4639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:25.196846 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7f387727-44d3-4a71-97c8-c0d078bf4639 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:25.198077 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7f387727-44d3-4a71-97c8-c0d078bf4639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:25.278169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 163/320] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:25.303099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f968fe-0556-4477-9667-62c98c3e7b20 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:25.380151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd94f9-8d82-45b3-a137-af17fbd9fcc6 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:25.380497 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd94f9-8d82-45b3-a137-af17fbd9fcc6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:25.380696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd94f9-8d82-45b3-a137-af17fbd9fcc6 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:25.382494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 164/321] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:39:25.389317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554950e6-6095-44a3-828f-899e5619e32f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:25.389613 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554950e6-6095-44a3-828f-899e5619e32f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:25.389819 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554950e6-6095-44a3-828f-899e5619e32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:25.410289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-554950e6-6095-44a3-828f-899e5619e32f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:25.412720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-554950e6-6095-44a3-828f-899e5619e32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:25.425425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f968fe-0556-4477-9667-62c98c3e7b20 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:25.455212 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:25.470895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdfca0-d6be-4f5a-9659-4b5662593854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:25.472851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdfca0-d6be-4f5a-9659-4b5662593854 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:25.472851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdfca0-d6be-4f5a-9659-4b5662593854 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:25.472851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdfca0-d6be-4f5a-9659-4b5662593854 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:25.474178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cdfca0-d6be-4f5a-9659-4b5662593854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:25.475797 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cdfca0-d6be-4f5a-9659-4b5662593854 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:25.481906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 158/322] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:25 2022] GET /identity/v3/domains/default => generated 200 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:25.483419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdfca0-d6be-4f5a-9659-4b5662593854 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:25.492836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 165/323] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:25 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:25.504722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:25.519112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554950e6-6095-44a3-828f-899e5619e32f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:25.587406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdfca0-d6be-4f5a-9659-4b5662593854 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:25.624220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554950e6-6095-44a3-828f-899e5619e32f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:25.626220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4114a3-15a4-4454-8735-0fc21ed1936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:25.627158 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4114a3-15a4-4454-8735-0fc21ed1936c None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:25.627522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4114a3-15a4-4454-8735-0fc21ed1936c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:25.627802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4114a3-15a4-4454-8735-0fc21ed1936c None admin] PATH_INFO: `/v3/groups` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:25.628487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4114a3-15a4-4454-8735-0fc21ed1936c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:25.630143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4114a3-15a4-4454-8735-0fc21ed1936c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:25.659026 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d848cfb-face-4570-bcd3-4e861051daeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:25.659822 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d848cfb-face-4570-bcd3-4e861051daeb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:25.660109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d848cfb-face-4570-bcd3-4e861051daeb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:25.660374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d848cfb-face-4570-bcd3-4e861051daeb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:25.671682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d848cfb-face-4570-bcd3-4e861051daeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-0d848cfb-face-4570-bcd3-4e861051daeb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:25.680447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:25.681580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 166/324] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:25.693337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d848cfb-face-4570-bcd3-4e861051daeb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:25.718135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:25.764260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 159/325] 10.222.0.121 () {60 vars in 1139 bytes} [Thu May 26 17:39:25 2022] POST /identity/v3/groups => generated 227 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:39:25.795148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d848cfb-face-4570-bcd3-4e861051daeb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:25.895246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d6d31-a102-43e5-911f-85cc340aba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:25.895947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d6d31-a102-43e5-911f-85cc340aba0a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:25.896221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d6d31-a102-43e5-911f-85cc340aba0a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:25.896494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d6d31-a102-43e5-911f-85cc340aba0a None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:25.897260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d6d31-a102-43e5-911f-85cc340aba0a None admin] RBAC: Adding query filter params (name=reader) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:25.897563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d6d31-a102-43e5-911f-85cc340aba0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:25.899059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d6d31-a102-43e5-911f-85cc340aba0a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:25.909372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 167/326] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:25 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:25.915511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4114a3-15a4-4454-8735-0fc21ed1936c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:26.006944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:26.028150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4114a3-15a4-4454-8735-0fc21ed1936c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:26.065866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f35f38-fc8a-46cb-980c-038c27f2b223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:26.067587 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f35f38-fc8a-46cb-980c-038c27f2b223 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:26.067883 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f35f38-fc8a-46cb-980c-038c27f2b223 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:26.068200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f35f38-fc8a-46cb-980c-038c27f2b223 None admin] PATH_INFO: `/v3/users/d1106a40711140aeba0f31c32c2c8edd` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:26.069656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f35f38-fc8a-46cb-980c-038c27f2b223 None admin] RBAC: Authorizing `identity:get_user(user_id=d1106a40711140aeba0f31c32c2c8edd)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:26.071268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f35f38-fc8a-46cb-980c-038c27f2b223 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:26.073261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 160/327] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:25 2022] GET /identity/v3/users/d1106a40711140aeba0f31c32c2c8edd => generated 294 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:26.078691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d6d31-a102-43e5-911f-85cc340aba0a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:26.108817 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc005d3-da15-4257-9b30-7f73ad89e65b None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:26.109105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc005d3-da15-4257-9b30-7f73ad89e65b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:26.109384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc005d3-da15-4257-9b30-7f73ad89e65b None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:26.111071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 161/328] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:39:26.117381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a0d64-5b74-466f-8f58-cac6988ef128 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:26.117727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a0d64-5b74-466f-8f58-cac6988ef128 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:26.118004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a0d64-5b74-466f-8f58-cac6988ef128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:26.147285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-8a8a0d64-5b74-466f-8f58-cac6988ef128 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:26.148599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-8a8a0d64-5b74-466f-8f58-cac6988ef128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:26.217228 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:26.230620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d6d31-a102-43e5-911f-85cc340aba0a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:26.235635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 162/329] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:26.259156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a0d64-5b74-466f-8f58-cac6988ef128 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:26.276220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c119a1e-428c-4799-a88c-1315257f6979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:26.277362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c119a1e-428c-4799-a88c-1315257f6979 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:26.277806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c119a1e-428c-4799-a88c-1315257f6979 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:26.278222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c119a1e-428c-4799-a88c-1315257f6979 None admin] PATH_INFO: `/v3/system/users/d1106a40711140aeba0f31c32c2c8edd/roles/6d1a0644f0bf45b4aa0f9fd0b3843261` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:26.282032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c119a1e-428c-4799-a88c-1315257f6979 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d1106a40711140aeba0f31c32c2c8edd, role_id=6d1a0644f0bf45b4aa0f9fd0b3843261)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:26.283664 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c119a1e-428c-4799-a88c-1315257f6979 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:26.349933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 168/330] 10.222.0.121 () {58 vars in 1339 bytes} [Thu May 26 17:39:26 2022] PUT /identity/v3/system/users/d1106a40711140aeba0f31c32c2c8edd/roles/6d1a0644f0bf45b4aa0f9fd0b3843261 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:39:26.353299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:26.386244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a0d64-5b74-466f-8f58-cac6988ef128 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:26.434087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df064a4-de41-4727-b51f-5317cbf92b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:26.435297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df064a4-de41-4727-b51f-5317cbf92b3e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:26.435748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df064a4-de41-4727-b51f-5317cbf92b3e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:26.436186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df064a4-de41-4727-b51f-5317cbf92b3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:26.453816 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df064a4-de41-4727-b51f-5317cbf92b3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-8df064a4-de41-4727-b51f-5317cbf92b3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:26.467320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:26.468466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:26.469425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:26.469425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:26.469425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:26.469425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:26.469425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:26.469425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:26.469425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:26.469425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:26.469425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:26.469425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:26.469425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:26.469425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:26.472024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 163/331] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:26.478773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c119a1e-428c-4799-a88c-1315257f6979 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:26.615835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c119a1e-428c-4799-a88c-1315257f6979 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:26.658479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd5d0d-6348-40a8-8428-40f9c444ff89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:26.659477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd5d0d-6348-40a8-8428-40f9c444ff89 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:26.659906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd5d0d-6348-40a8-8428-40f9c444ff89 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:26.660340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd5d0d-6348-40a8-8428-40f9c444ff89 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:26.661829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd5d0d-6348-40a8-8428-40f9c444ff89 None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:26.662335 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd5d0d-6348-40a8-8428-40f9c444ff89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:26.664893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd5d0d-6348-40a8-8428-40f9c444ff89 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:26.679461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 169/332] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:26 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:26.686556 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df064a4-de41-4727-b51f-5317cbf92b3e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:26.809207 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df064a4-de41-4727-b51f-5317cbf92b3e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:26.847377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc522ef6-4c6a-4a09-93ba-d1e99d62302f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:26.848821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc522ef6-4c6a-4a09-93ba-d1e99d62302f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:26.849152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc522ef6-4c6a-4a09-93ba-d1e99d62302f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:26.849433 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc522ef6-4c6a-4a09-93ba-d1e99d62302f None admin] PATH_INFO: `/v3/users/dc4b60c90c48454383c96a4977c507e8` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:26.850707 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc522ef6-4c6a-4a09-93ba-d1e99d62302f None admin] RBAC: Authorizing `identity:get_user(user_id=dc4b60c90c48454383c96a4977c507e8)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:26.852293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc522ef6-4c6a-4a09-93ba-d1e99d62302f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:26.854262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 164/333] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:26 2022] GET /identity/v3/users/dc4b60c90c48454383c96a4977c507e8 => generated 294 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:26.860031 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd5d0d-6348-40a8-8428-40f9c444ff89 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:26.903968 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:27.015546 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd5d0d-6348-40a8-8428-40f9c444ff89 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:27.072520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea93c4-2f2c-449b-a583-a4c57a4d078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:27.073573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea93c4-2f2c-449b-a583-a4c57a4d078a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:27.074055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea93c4-2f2c-449b-a583-a4c57a4d078a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:27.074540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea93c4-2f2c-449b-a583-a4c57a4d078a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:27.075622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea93c4-2f2c-449b-a583-a4c57a4d078a None admin] RBAC: Adding query filter params (role.id=e87c07d8d70d46d89a2c4f7246edeff6, user.id=dc4b60c90c48454383c96a4977c507e8, scope.system=all) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:27.076177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea93c4-2f2c-449b-a583-a4c57a4d078a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:27.077844 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea93c4-2f2c-449b-a583-a4c57a4d078a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:27.094927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 170/334] 10.222.0.121 () {56 vars in 1313 bytes} [Thu May 26 17:39:26 2022] GET /identity/v3/role_assignments?user.id=dc4b60c90c48454383c96a4977c507e8&scope.system=all&role.id=e87c07d8d70d46d89a2c4f7246edeff6 => generated 512 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:28.387157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0710fb-9b91-467f-b41d-76da4f236f0f None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:28.387944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0710fb-9b91-467f-b41d-76da4f236f0f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:28.387944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0710fb-9b91-467f-b41d-76da4f236f0f None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:28.389362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 165/335] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:28.394544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b52e3d-a225-47c6-85b0-613ad1aea22e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:28.395311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b52e3d-a225-47c6-85b0-613ad1aea22e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:28.395434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b52e3d-a225-47c6-85b0-613ad1aea22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:28.421344 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-32b52e3d-a225-47c6-85b0-613ad1aea22e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:28.422546 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-32b52e3d-a225-47c6-85b0-613ad1aea22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:28.514454 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 171/336] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:28.552146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0710fb-9b91-467f-b41d-76da4f236f0f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:28.556387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:28.685432 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0710fb-9b91-467f-b41d-76da4f236f0f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:28.731744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1de77-0579-4a24-bcb0-cbd73a639cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:28.732909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1de77-0579-4a24-bcb0-cbd73a639cda None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:28.733438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1de77-0579-4a24-bcb0-cbd73a639cda None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:28.733908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1de77-0579-4a24-bcb0-cbd73a639cda None admin] PATH_INFO: `/v3/roles/member` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:28.749757 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1de77-0579-4a24-bcb0-cbd73a639cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d6b1de77-0579-4a24-bcb0-cbd73a639cda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:28.758098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:28.759042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:28.759882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:28.759882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:28.759882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:28.759882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:28.759882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:28.759882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:28.759882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:28.759882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:28.759882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:28.759882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:28.759882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:28.759882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:28.762180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 166/337] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:28.768245 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b52e3d-a225-47c6-85b0-613ad1aea22e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:28.784167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:28.934338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b52e3d-a225-47c6-85b0-613ad1aea22e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:28.983344 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cea6e-ac91-42df-aec0-e0171390a922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:28.983955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cea6e-ac91-42df-aec0-e0171390a922 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:28.984239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cea6e-ac91-42df-aec0-e0171390a922 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:28.984514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cea6e-ac91-42df-aec0-e0171390a922 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:28.985272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cea6e-ac91-42df-aec0-e0171390a922 None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:28.985579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cea6e-ac91-42df-aec0-e0171390a922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:28.987007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cea6e-ac91-42df-aec0-e0171390a922 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:29.003171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 172/338] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:28 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:29.009383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1de77-0579-4a24-bcb0-cbd73a639cda None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:29.083536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:29.145665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1de77-0579-4a24-bcb0-cbd73a639cda None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:29.180751 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355881ef-3f24-47d0-b043-5adb2dc6bef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:29.181422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355881ef-3f24-47d0-b043-5adb2dc6bef1 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:29.181702 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355881ef-3f24-47d0-b043-5adb2dc6bef1 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:29.181976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355881ef-3f24-47d0-b043-5adb2dc6bef1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:29.272791 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355881ef-3f24-47d0-b043-5adb2dc6bef1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:29.274181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355881ef-3f24-47d0-b043-5adb2dc6bef1 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:29.350138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:29.400948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 167/339] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:29 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:29.406520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cea6e-ac91-42df-aec0-e0171390a922 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:29.459578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deff1fb-be80-4f1a-ae78-cb20e46187f0 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:29.459800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deff1fb-be80-4f1a-ae78-cb20e46187f0 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:29.460064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deff1fb-be80-4f1a-ae78-cb20e46187f0 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:29.461446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 168/340] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:39:29.466947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe722c-8363-4e74-ab89-d7e368f6c200 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:29.467229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe722c-8363-4e74-ab89-d7e368f6c200 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:29.468198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe722c-8363-4e74-ab89-d7e368f6c200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:29.489510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-52fe722c-8363-4e74-ab89-d7e368f6c200 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:29.490700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-52fe722c-8363-4e74-ab89-d7e368f6c200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:29.529993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:29.580850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 169/341] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:29.602942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cea6e-ac91-42df-aec0-e0171390a922 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:29.617214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe722c-8363-4e74-ab89-d7e368f6c200 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:29.670184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f5325-92ea-419a-9a67-feaa806b41f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:29.671096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f5325-92ea-419a-9a67-feaa806b41f5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:29.671373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f5325-92ea-419a-9a67-feaa806b41f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:29.671650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f5325-92ea-419a-9a67-feaa806b41f5 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:29.672842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f5325-92ea-419a-9a67-feaa806b41f5 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:29.674315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f5325-92ea-419a-9a67-feaa806b41f5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:29.676873 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 173/342] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:29 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:29.688187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f5325-92ea-419a-9a67-feaa806b41f5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:29.741701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:29.745238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe722c-8363-4e74-ab89-d7e368f6c200 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:29.781151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8282c8-3c04-41a9-af2f-3fa8ea09c915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:29.781886 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8282c8-3c04-41a9-af2f-3fa8ea09c915 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:29.782170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8282c8-3c04-41a9-af2f-3fa8ea09c915 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:29.782434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8282c8-3c04-41a9-af2f-3fa8ea09c915 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:29.803163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8282c8-3c04-41a9-af2f-3fa8ea09c915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-4e8282c8-3c04-41a9-af2f-3fa8ea09c915 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:29.813200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:29.814149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:29.814966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:29.814966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:29.814966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:29.814966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:29.814966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:29.814966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:29.814966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:29.814966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:29.814966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:29.814966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:29.814966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:29.814966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:29.815589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 170/343] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:39:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:29.821090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8282c8-3c04-41a9-af2f-3fa8ea09c915 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:29.912630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f5325-92ea-419a-9a67-feaa806b41f5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:29.988341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbf411-3ff1-4eb2-8a41-4191552af0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:29.989954 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbf411-3ff1-4eb2-8a41-4191552af0b7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:29.990233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbf411-3ff1-4eb2-8a41-4191552af0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:29.990506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbf411-3ff1-4eb2-8a41-4191552af0b7 None admin] PATH_INFO: `/v3/groups/3bb70362290945a6bcba24363af6df8b` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:29.995203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:29.997754 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8282c8-3c04-41a9-af2f-3fa8ea09c915 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:30.004285 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbf411-3ff1-4eb2-8a41-4191552af0b7 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb70362290945a6bcba24363af6df8b)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:30.005778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbf411-3ff1-4eb2-8a41-4191552af0b7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:30.015056 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 174/344] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:29 2022] GET /identity/v3/groups/3bb70362290945a6bcba24363af6df8b => generated 227 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:30.016027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbf411-3ff1-4eb2-8a41-4191552af0b7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:30.056240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426b481-099d-4f82-98d2-c349d31df4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:30.056856 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426b481-099d-4f82-98d2-c349d31df4d1 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:30.057140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426b481-099d-4f82-98d2-c349d31df4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:30.057417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426b481-099d-4f82-98d2-c349d31df4d1 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:30.058179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426b481-099d-4f82-98d2-c349d31df4d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:30.058486 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426b481-099d-4f82-98d2-c349d31df4d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:30.059923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426b481-099d-4f82-98d2-c349d31df4d1 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:30.069151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 171/345] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:29 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:30.079836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426b481-099d-4f82-98d2-c349d31df4d1 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:30.111222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:30.228793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbf411-3ff1-4eb2-8a41-4191552af0b7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:30.237083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426b481-099d-4f82-98d2-c349d31df4d1 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:30.276123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e00b65-27ec-4bfd-a9e1-db99b407700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:30.276785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e00b65-27ec-4bfd-a9e1-db99b407700d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:30.277065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e00b65-27ec-4bfd-a9e1-db99b407700d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:30.277338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e00b65-27ec-4bfd-a9e1-db99b407700d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:30.326349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1ceeb-6e26-42fe-82c5-f55b26ceddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:30.326971 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1ceeb-6e26-42fe-82c5-f55b26ceddd1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:30.327254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1ceeb-6e26-42fe-82c5-f55b26ceddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:30.327528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1ceeb-6e26-42fe-82c5-f55b26ceddd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:30.328377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1ceeb-6e26-42fe-82c5-f55b26ceddd1 None admin] RBAC: Adding query filter params (group.id=3bb70362290945a6bcba24363af6df8b, role.id=e87c07d8d70d46d89a2c4f7246edeff6, scope.project.id=8611e940b0fa4f8d910441467b31bdce) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:30.328751 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1ceeb-6e26-42fe-82c5-f55b26ceddd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:30.330260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1ceeb-6e26-42fe-82c5-f55b26ceddd1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:30.346078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 175/346] 10.222.0.121 () {56 vars in 1381 bytes} [Thu May 26 17:39:30 2022] GET /identity/v3/role_assignments?group.id=3bb70362290945a6bcba24363af6df8b&scope.project.id=8611e940b0fa4f8d910441467b31bdce&role.id=e87c07d8d70d46d89a2c4f7246edeff6 => generated 261 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:30.347468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11341a-70cb-4cf5-a72e-bfa1eb8fd288 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:30.347789 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11341a-70cb-4cf5-a72e-bfa1eb8fd288 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:30.348138 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11341a-70cb-4cf5-a72e-bfa1eb8fd288 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:30.349684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 176/347] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:30.355833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2adee-9c51-466a-bf23-8cfc48d8f394 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:30.356172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2adee-9c51-466a-bf23-8cfc48d8f394 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:30.356283 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e00b65-27ec-4bfd-a9e1-db99b407700d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:30.356461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2adee-9c51-466a-bf23-8cfc48d8f394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:30.357827 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e00b65-27ec-4bfd-a9e1-db99b407700d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:30.377580 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:30.388475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e1f2adee-9c51-466a-bf23-8cfc48d8f394 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:30.389811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e1f2adee-9c51-466a-bf23-8cfc48d8f394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:30.471357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 172/348] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:30 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:39:30.480032 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e00b65-27ec-4bfd-a9e1-db99b407700d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:30.546009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:30.578401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 177/349] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:30.625985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2adee-9c51-466a-bf23-8cfc48d8f394 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:30.630110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e00b65-27ec-4bfd-a9e1-db99b407700d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:30.667608 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:30.685018 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c1f71-a644-4938-96c5-ec79dbd64380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:30.685909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c1f71-a644-4938-96c5-ec79dbd64380 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:30.688623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c1f71-a644-4938-96c5-ec79dbd64380 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:30.688623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c1f71-a644-4938-96c5-ec79dbd64380 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:30.688623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c1f71-a644-4938-96c5-ec79dbd64380 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:30.689260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c1f71-a644-4938-96c5-ec79dbd64380 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:30.691372 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 173/350] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:30 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:30.698353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c1f71-a644-4938-96c5-ec79dbd64380 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:30.837803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c1f71-a644-4938-96c5-ec79dbd64380 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:30.868007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2adee-9c51-466a-bf23-8cfc48d8f394 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:30.880906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86cb6d-e3ed-47ba-b956-7d133560d6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:30.881875 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86cb6d-e3ed-47ba-b956-7d133560d6c0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:30.882166 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86cb6d-e3ed-47ba-b956-7d133560d6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:30.882444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86cb6d-e3ed-47ba-b956-7d133560d6c0 None admin] PATH_INFO: `/v3/groups/3bb70362290945a6bcba24363af6df8b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:30.883729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86cb6d-e3ed-47ba-b956-7d133560d6c0 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb70362290945a6bcba24363af6df8b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:30.885203 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86cb6d-e3ed-47ba-b956-7d133560d6c0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:30.888249 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 174/351] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:30 2022] GET /identity/v3/groups/3bb70362290945a6bcba24363af6df8b => generated 227 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:30.888997 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51110052-c2c1-4c71-bd7d-e65370b2f4f8 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:30.889287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51110052-c2c1-4c71-bd7d-e65370b2f4f8 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:30.889575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51110052-c2c1-4c71-bd7d-e65370b2f4f8 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:30.891218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 175/352] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:30.893792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51110052-c2c1-4c71-bd7d-e65370b2f4f8 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:30.955224 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:30.971615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d2798-c224-431d-add7-35c6931a4f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:30.972442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d2798-c224-431d-add7-35c6931a4f76 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:30.972830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d2798-c224-431d-add7-35c6931a4f76 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:30.973106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d2798-c224-431d-add7-35c6931a4f76 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:31.007540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d2798-c224-431d-add7-35c6931a4f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:31.008417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51110052-c2c1-4c71-bd7d-e65370b2f4f8 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-1c9d2798-c224-431d-add7-35c6931a4f76 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:31.015622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:31.016799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:31.017603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 178/353] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:31.018644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91d1ba-d7ee-44ca-9ac3-08e8de911720 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:31.018644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91d1ba-d7ee-44ca-9ac3-08e8de911720 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:31.023044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91d1ba-d7ee-44ca-9ac3-08e8de911720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:31.055908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9c91d1ba-d7ee-44ca-9ac3-08e8de911720 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:31.057156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9c91d1ba-d7ee-44ca-9ac3-08e8de911720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:31.069045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736bc1d-f292-4f94-9494-9fa85789a2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:31.069734 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736bc1d-f292-4f94-9494-9fa85789a2b5 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:31.070014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736bc1d-f292-4f94-9494-9fa85789a2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:31.070287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736bc1d-f292-4f94-9494-9fa85789a2b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:31.071168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736bc1d-f292-4f94-9494-9fa85789a2b5 None admin] RBAC: Adding query filter params (group.id=3bb70362290945a6bcba24363af6df8b, role.id=582f59fc05ce474287c6fdc503f9ac82, scope.project.id=8611e940b0fa4f8d910441467b31bdce) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:31.071483 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736bc1d-f292-4f94-9494-9fa85789a2b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:31.072976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736bc1d-f292-4f94-9494-9fa85789a2b5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:31.095031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 176/354] 10.222.0.121 () {56 vars in 1381 bytes} [Thu May 26 17:39:30 2022] GET /identity/v3/role_assignments?group.id=3bb70362290945a6bcba24363af6df8b&scope.project.id=8611e940b0fa4f8d910441467b31bdce&role.id=582f59fc05ce474287c6fdc503f9ac82 => generated 261 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:31.095415 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736bc1d-f292-4f94-9494-9fa85789a2b5 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:31.152356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:31.182087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 179/355] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:31.219040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91d1ba-d7ee-44ca-9ac3-08e8de911720 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:31.300596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736bc1d-f292-4f94-9494-9fa85789a2b5 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:31.331969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:31.364017 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79595ad4-248e-4979-9a70-5fd0eaad1655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:31.364795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79595ad4-248e-4979-9a70-5fd0eaad1655 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:31.365139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79595ad4-248e-4979-9a70-5fd0eaad1655 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:31.365416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79595ad4-248e-4979-9a70-5fd0eaad1655 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:31.366308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79595ad4-248e-4979-9a70-5fd0eaad1655 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:31.366617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79595ad4-248e-4979-9a70-5fd0eaad1655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:31.368394 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79595ad4-248e-4979-9a70-5fd0eaad1655 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:31.378384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 177/356] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:31 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:31.381350 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635e99f-8585-4a56-b4dd-93a318c53731 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:31.381625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635e99f-8585-4a56-b4dd-93a318c53731 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:31.381907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635e99f-8585-4a56-b4dd-93a318c53731 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:31.383695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 178/357] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:31 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:31.385871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635e99f-8585-4a56-b4dd-93a318c53731 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:31.451603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91d1ba-d7ee-44ca-9ac3-08e8de911720 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:31.514504 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3e3f3-6615-4e91-98cd-7ba2d0860d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:31.515810 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3e3f3-6615-4e91-98cd-7ba2d0860d74 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:31.516288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3e3f3-6615-4e91-98cd-7ba2d0860d74 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:31.517132 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3e3f3-6615-4e91-98cd-7ba2d0860d74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:31.538111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3e3f3-6615-4e91-98cd-7ba2d0860d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:31.546703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635e99f-8585-4a56-b4dd-93a318c53731 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-a5f3e3f3-6615-4e91-98cd-7ba2d0860d74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:31.556579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:31.558807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:31.560348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:31.560348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:31.560348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:31.560348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:31.560348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:31.560348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:31.560348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:31.560348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:31.560348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:31.560348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:31.560348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:31.560348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:31.563161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 180/358] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:31.565259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc661d40-23bd-4d88-8e40-8678e96464be None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:31.565738 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc661d40-23bd-4d88-8e40-8678e96464be None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:31.566214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc661d40-23bd-4d88-8e40-8678e96464be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:31.594133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bc661d40-23bd-4d88-8e40-8678e96464be None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:31.595520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bc661d40-23bd-4d88-8e40-8678e96464be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:31.605183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867d3ed-1f31-4d0f-b10f-8f0fa85aa72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:31.606424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867d3ed-1f31-4d0f-b10f-8f0fa85aa72d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:31.606945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867d3ed-1f31-4d0f-b10f-8f0fa85aa72d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:31.607448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867d3ed-1f31-4d0f-b10f-8f0fa85aa72d None admin] PATH_INFO: `/v3/users/d1106a40711140aeba0f31c32c2c8edd` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:31.608876 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867d3ed-1f31-4d0f-b10f-8f0fa85aa72d None admin] RBAC: Authorizing `identity:get_user(user_id=d1106a40711140aeba0f31c32c2c8edd)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:31.610488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867d3ed-1f31-4d0f-b10f-8f0fa85aa72d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:31.615014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 179/359] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:39:31 2022] GET /identity/v3/users/d1106a40711140aeba0f31c32c2c8edd => generated 294 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:31.616083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867d3ed-1f31-4d0f-b10f-8f0fa85aa72d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:31.654923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:31.673555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:31.693192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 181/360] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:31.695766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc661d40-23bd-4d88-8e40-8678e96464be None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:31.801888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867d3ed-1f31-4d0f-b10f-8f0fa85aa72d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:31.850676 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc661d40-23bd-4d88-8e40-8678e96464be None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:31.890291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee19625-7ba4-47a9-9ac6-be627af53a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:31.891342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee19625-7ba4-47a9-9ac6-be627af53a8f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:31.892213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee19625-7ba4-47a9-9ac6-be627af53a8f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:31.892894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee19625-7ba4-47a9-9ac6-be627af53a8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:31.893926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee19625-7ba4-47a9-9ac6-be627af53a8f None admin] RBAC: Adding query filter params (role.id=6d1a0644f0bf45b4aa0f9fd0b3843261, user.id=d1106a40711140aeba0f31c32c2c8edd, scope.system=all) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:31.894432 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee19625-7ba4-47a9-9ac6-be627af53a8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:31.897224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee19625-7ba4-47a9-9ac6-be627af53a8f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:31.911385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 182/361] 10.222.0.121 () {56 vars in 1313 bytes} [Thu May 26 17:39:31 2022] GET /identity/v3/role_assignments?user.id=d1106a40711140aeba0f31c32c2c8edd&scope.system=all&role.id=6d1a0644f0bf45b4aa0f9fd0b3843261 => generated 512 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:31.911927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e561ca-ffa9-40e7-8e67-63d21211973a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:31.912639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e561ca-ffa9-40e7-8e67-63d21211973a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:31.912927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e561ca-ffa9-40e7-8e67-63d21211973a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:31.913206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e561ca-ffa9-40e7-8e67-63d21211973a None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:31.913711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee19625-7ba4-47a9-9ac6-be627af53a8f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:31.913973 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e561ca-ffa9-40e7-8e67-63d21211973a None admin] RBAC: Adding query filter params (name=member) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:31.914285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e561ca-ffa9-40e7-8e67-63d21211973a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:31.915722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e561ca-ffa9-40e7-8e67-63d21211973a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:31.933383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 180/362] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:31 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:31.938968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e561ca-ffa9-40e7-8e67-63d21211973a None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:31.978753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:32.070073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e561ca-ffa9-40e7-8e67-63d21211973a None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:32.080740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:32.116060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df35876-bf1c-41c3-99cd-c6303380fda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:32.117239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df35876-bf1c-41c3-99cd-c6303380fda3 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:32.117845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df35876-bf1c-41c3-99cd-c6303380fda3 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:32.118318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df35876-bf1c-41c3-99cd-c6303380fda3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:32.180888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee19625-7ba4-47a9-9ac6-be627af53a8f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:32.199380 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df35876-bf1c-41c3-99cd-c6303380fda3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:32.201304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df35876-bf1c-41c3-99cd-c6303380fda3 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:32.266594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bee47-a391-4b6b-b7b9-ae52ff85da20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:32.267662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bee47-a391-4b6b-b7b9-ae52ff85da20 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:32.267771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bee47-a391-4b6b-b7b9-ae52ff85da20 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:32.268011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bee47-a391-4b6b-b7b9-ae52ff85da20 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:32.281907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bee47-a391-4b6b-b7b9-ae52ff85da20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-5b6bee47-a391-4b6b-b7b9-ae52ff85da20 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:32.292615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:32.293602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:32.294405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 183/363] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:39:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:32.295862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63a63d-fc1e-4aa9-a04d-a3ec462e9d95 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:32.295862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63a63d-fc1e-4aa9-a04d-a3ec462e9d95 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:32.295862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63a63d-fc1e-4aa9-a04d-a3ec462e9d95 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:32.297464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 184/364] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:32.305650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d981e69-97a7-4299-ba7c-20dd9157799f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:32.305650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d981e69-97a7-4299-ba7c-20dd9157799f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:32.305650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d981e69-97a7-4299-ba7c-20dd9157799f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:32.324236 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4d981e69-97a7-4299-ba7c-20dd9157799f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:32.325599 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4d981e69-97a7-4299-ba7c-20dd9157799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:32.326210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 181/365] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:31 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:39:32.327293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df35876-bf1c-41c3-99cd-c6303380fda3 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:32.338739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:32.411713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 185/366] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:32.413015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d981e69-97a7-4299-ba7c-20dd9157799f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:32.446065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df35876-bf1c-41c3-99cd-c6303380fda3 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:32.449813 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:32.489316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841fd47-eae0-4cf9-b061-b35653a4dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:32.490201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841fd47-eae0-4cf9-b061-b35653a4dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:32.490483 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841fd47-eae0-4cf9-b061-b35653a4dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:32.490752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841fd47-eae0-4cf9-b061-b35653a4dfc6 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:32.491560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841fd47-eae0-4cf9-b061-b35653a4dfc6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:32.491875 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841fd47-eae0-4cf9-b061-b35653a4dfc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:32.493236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841fd47-eae0-4cf9-b061-b35653a4dfc6 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:32.502080 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 182/367] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:32 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:32.502727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841fd47-eae0-4cf9-b061-b35653a4dfc6 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:32.532939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d981e69-97a7-4299-ba7c-20dd9157799f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:32.572517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5cd76-074d-40d0-87f7-d09c102dbda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:32.573488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5cd76-074d-40d0-87f7-d09c102dbda7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:32.573772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5cd76-074d-40d0-87f7-d09c102dbda7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:32.574044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5cd76-074d-40d0-87f7-d09c102dbda7 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:32.575261 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5cd76-074d-40d0-87f7-d09c102dbda7 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:32.576950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5cd76-074d-40d0-87f7-d09c102dbda7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:32.578656 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 186/368] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:32 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:32.579524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5cd76-074d-40d0-87f7-d09c102dbda7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:32.584993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:32.619680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841fd47-eae0-4cf9-b061-b35653a4dfc6 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:32.655606 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704b18b-4978-4668-bcc8-bd85a9fbf8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:32.656414 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704b18b-4978-4668-bcc8-bd85a9fbf8eb None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:32.656691 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704b18b-4978-4668-bcc8-bd85a9fbf8eb None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:32.656955 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704b18b-4978-4668-bcc8-bd85a9fbf8eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:32.669670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704b18b-4978-4668-bcc8-bd85a9fbf8eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-e704b18b-4978-4668-bcc8-bd85a9fbf8eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:32.677511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:32.678449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:32.679253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 183/369] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:39:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:32.679919 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704b18b-4978-4668-bcc8-bd85a9fbf8eb None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:32.702387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5cd76-074d-40d0-87f7-d09c102dbda7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:32.729373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:32.746015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acff067-0146-47bc-9181-82428b7d0d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:32.746850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acff067-0146-47bc-9181-82428b7d0d8f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:32.747128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acff067-0146-47bc-9181-82428b7d0d8f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:32.747404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acff067-0146-47bc-9181-82428b7d0d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:32.789671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704b18b-4978-4668-bcc8-bd85a9fbf8eb None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:32.823163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:32.829477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ac004-dd61-4364-aa9f-907993dc6b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:32.830214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acff067-0146-47bc-9181-82428b7d0d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:32.830398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ac004-dd61-4364-aa9f-907993dc6b01 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:32.830645 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ac004-dd61-4364-aa9f-907993dc6b01 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:32.830947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ac004-dd61-4364-aa9f-907993dc6b01 None admin] PATH_INFO: `/v3/groups/3bb70362290945a6bcba24363af6df8b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:32.831837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acff067-0146-47bc-9181-82428b7d0d8f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:32.832580 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ac004-dd61-4364-aa9f-907993dc6b01 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb70362290945a6bcba24363af6df8b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:32.834297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ac004-dd61-4364-aa9f-907993dc6b01 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:32.836082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 184/370] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:32 2022] GET /identity/v3/groups/3bb70362290945a6bcba24363af6df8b => generated 227 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:32.836868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ac004-dd61-4364-aa9f-907993dc6b01 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:32.938464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:32.954459 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 187/371] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:32 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:32.954863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ac004-dd61-4364-aa9f-907993dc6b01 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:32.954972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acff067-0146-47bc-9181-82428b7d0d8f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:33.002256 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca755af5-3f12-4fa7-b475-cac25c61db59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:33.003224 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca755af5-3f12-4fa7-b475-cac25c61db59 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:33.003632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca755af5-3f12-4fa7-b475-cac25c61db59 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:33.004064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca755af5-3f12-4fa7-b475-cac25c61db59 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:33.005174 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca755af5-3f12-4fa7-b475-cac25c61db59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:33.005592 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca755af5-3f12-4fa7-b475-cac25c61db59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:33.007718 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca755af5-3f12-4fa7-b475-cac25c61db59 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:33.021686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 185/372] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:32 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:33.022931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca755af5-3f12-4fa7-b475-cac25c61db59 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:33.098335 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:33.100334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acff067-0146-47bc-9181-82428b7d0d8f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:33.143166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002a95d-ebb8-4b97-bae2-0234c027b62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:33.143830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002a95d-ebb8-4b97-bae2-0234c027b62b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:33.144108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002a95d-ebb8-4b97-bae2-0234c027b62b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:33.144370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002a95d-ebb8-4b97-bae2-0234c027b62b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:33.145475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002a95d-ebb8-4b97-bae2-0234c027b62b None admin] RBAC: Adding query filter params (group.id=3bb70362290945a6bcba24363af6df8b, role.id=e87c07d8d70d46d89a2c4f7246edeff6, scope.project.id=77149f8497f3433c8a1c236067c9eb62) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:33.145950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002a95d-ebb8-4b97-bae2-0234c027b62b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:33.147564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002a95d-ebb8-4b97-bae2-0234c027b62b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:33.153731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca755af5-3f12-4fa7-b475-cac25c61db59 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:33.154064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 188/373] 10.222.0.121 () {56 vars in 1381 bytes} [Thu May 26 17:39:32 2022] GET /identity/v3/role_assignments?group.id=3bb70362290945a6bcba24363af6df8b&scope.project.id=77149f8497f3433c8a1c236067c9eb62&role.id=e87c07d8d70d46d89a2c4f7246edeff6 => generated 261 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:33.154943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002a95d-ebb8-4b97-bae2-0234c027b62b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:33.213784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfc315-5cf0-44f0-83b3-2d3295717f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:33.214604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc315-5cf0-44f0-83b3-2d3295717f2a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:33.215025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc315-5cf0-44f0-83b3-2d3295717f2a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:33.215307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc315-5cf0-44f0-83b3-2d3295717f2a None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:33.219853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfc315-5cf0-44f0-83b3-2d3295717f2a None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:33.221339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfc315-5cf0-44f0-83b3-2d3295717f2a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:33.222384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:33.224498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 186/374] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:33 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:33.242599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfc315-5cf0-44f0-83b3-2d3295717f2a None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:33.282008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002a95d-ebb8-4b97-bae2-0234c027b62b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:33.325039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba38fd-6a99-4c46-b231-845dca8713c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:33.325710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba38fd-6a99-4c46-b231-845dca8713c4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:33.325988 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba38fd-6a99-4c46-b231-845dca8713c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:33.326307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba38fd-6a99-4c46-b231-845dca8713c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:33.403683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba38fd-6a99-4c46-b231-845dca8713c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:33.405690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba38fd-6a99-4c46-b231-845dca8713c4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:33.443983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfc315-5cf0-44f0-83b3-2d3295717f2a None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:33.483297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:33.492531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:33.510986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee119126-f599-41ef-96af-f1f343efc149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:33.511901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee119126-f599-41ef-96af-f1f343efc149 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:33.512182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee119126-f599-41ef-96af-f1f343efc149 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:33.512494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee119126-f599-41ef-96af-f1f343efc149 None admin] PATH_INFO: `/v3/groups/3bb70362290945a6bcba24363af6df8b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:33.515095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee119126-f599-41ef-96af-f1f343efc149 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb70362290945a6bcba24363af6df8b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:33.516586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee119126-f599-41ef-96af-f1f343efc149 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:33.518873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 187/375] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:33 2022] GET /identity/v3/groups/3bb70362290945a6bcba24363af6df8b => generated 227 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:33.521146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a553991-ee84-4490-b338-83f783c93b93 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:33.521427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a553991-ee84-4490-b338-83f783c93b93 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:33.521663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a553991-ee84-4490-b338-83f783c93b93 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:33.528477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 188/376] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:33 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:33.529360 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a553991-ee84-4490-b338-83f783c93b93 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:33.532476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 189/377] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:33 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:33.533949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5adbb4-99df-437a-be2e-f2013a472860 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:33.534238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5adbb4-99df-437a-be2e-f2013a472860 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:33.534511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5adbb4-99df-437a-be2e-f2013a472860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:33.555742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2f5adbb4-99df-437a-be2e-f2013a472860 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:33.556963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2f5adbb4-99df-437a-be2e-f2013a472860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:33.638857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 190/378] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:33.639563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5adbb4-99df-437a-be2e-f2013a472860 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:33.698038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a553991-ee84-4490-b338-83f783c93b93 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:33.742027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4204ba-14af-46fc-a007-a0f41e6f90da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:33.742617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4204ba-14af-46fc-a007-a0f41e6f90da None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:33.742922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4204ba-14af-46fc-a007-a0f41e6f90da None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:33.743197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4204ba-14af-46fc-a007-a0f41e6f90da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:33.744035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4204ba-14af-46fc-a007-a0f41e6f90da None admin] RBAC: Adding query filter params (group.id=3bb70362290945a6bcba24363af6df8b, role.id=582f59fc05ce474287c6fdc503f9ac82, scope.project.id=77149f8497f3433c8a1c236067c9eb62) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:33.744347 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4204ba-14af-46fc-a007-a0f41e6f90da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:33.746297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4204ba-14af-46fc-a007-a0f41e6f90da None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:33.754110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 189/379] 10.222.0.121 () {56 vars in 1381 bytes} [Thu May 26 17:39:33 2022] GET /identity/v3/role_assignments?group.id=3bb70362290945a6bcba24363af6df8b&scope.project.id=77149f8497f3433c8a1c236067c9eb62&role.id=582f59fc05ce474287c6fdc503f9ac82 => generated 261 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:33.755109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4204ba-14af-46fc-a007-a0f41e6f90da None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:33.760918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:33.795602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5adbb4-99df-437a-be2e-f2013a472860 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:33.829516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:33.848739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1f7ca-5afe-43ea-9619-0536e9c82f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:33.850133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1f7ca-5afe-43ea-9619-0536e9c82f8d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:33.850609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1f7ca-5afe-43ea-9619-0536e9c82f8d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:33.851100 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1f7ca-5afe-43ea-9619-0536e9c82f8d None admin] PATH_INFO: `/v3/projects/a01a22dec10b499785fdfbdf0a6fcb47` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:33.852957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1f7ca-5afe-43ea-9619-0536e9c82f8d None admin] RBAC: Authorizing `identity:get_project(project_id=a01a22dec10b499785fdfbdf0a6fcb47)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:33.854795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1f7ca-5afe-43ea-9619-0536e9c82f8d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:33.856971 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 191/380] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:33 2022] GET /identity/v3/projects/a01a22dec10b499785fdfbdf0a6fcb47 => generated 345 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:33.866029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1f7ca-5afe-43ea-9619-0536e9c82f8d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:33.888037 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4204ba-14af-46fc-a007-a0f41e6f90da None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:33.936781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd35bd-5871-4140-a324-f110f791f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:33.937023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd35bd-5871-4140-a324-f110f791f97a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:33.937283 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd35bd-5871-4140-a324-f110f791f97a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:33.937563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd35bd-5871-4140-a324-f110f791f97a None admin] PATH_INFO: `/v3/roles/member` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:33.956190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd35bd-5871-4140-a324-f110f791f97a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d7dd35bd-5871-4140-a324-f110f791f97a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:33.966804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:33.967833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:33.968615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 190/381] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:33.974789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd35bd-5871-4140-a324-f110f791f97a None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:34.014594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1f7ca-5afe-43ea-9619-0536e9c82f8d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:34.084777 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5011e93-13fc-4d9c-9851-788d3dde43ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:34.085675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5011e93-13fc-4d9c-9851-788d3dde43ee None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:34.085953 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5011e93-13fc-4d9c-9851-788d3dde43ee None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:34.086225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5011e93-13fc-4d9c-9851-788d3dde43ee None admin] PATH_INFO: `/v3/groups/99468c055e7d412b907043393618a43f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:34.103154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5011e93-13fc-4d9c-9851-788d3dde43ee None admin] RBAC: Authorizing `identity:get_group(group_id=99468c055e7d412b907043393618a43f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:34.106206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5011e93-13fc-4d9c-9851-788d3dde43ee None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:34.106357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 192/382] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:33 2022] GET /identity/v3/groups/99468c055e7d412b907043393618a43f => generated 230 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:34.118759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5011e93-13fc-4d9c-9851-788d3dde43ee None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:34.157102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:34.174749 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd35bd-5871-4140-a324-f110f791f97a None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:34.201883 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:34.233453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf96e6-ae0b-41db-9746-5d7740dedc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:34.234163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf96e6-ae0b-41db-9746-5d7740dedc07 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:34.234426 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf96e6-ae0b-41db-9746-5d7740dedc07 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:34.234698 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf96e6-ae0b-41db-9746-5d7740dedc07 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:34.235482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf96e6-ae0b-41db-9746-5d7740dedc07 None admin] RBAC: Adding query filter params (name=member) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:34.235786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf96e6-ae0b-41db-9746-5d7740dedc07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:34.237151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf96e6-ae0b-41db-9746-5d7740dedc07 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:34.258175 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 191/383] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:33 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:34.267761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5011e93-13fc-4d9c-9851-788d3dde43ee None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:34.284467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf96e6-ae0b-41db-9746-5d7740dedc07 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:34.335001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5a239-48ff-478d-a202-b5d03e510f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:34.336130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5a239-48ff-478d-a202-b5d03e510f4f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:34.336633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5a239-48ff-478d-a202-b5d03e510f4f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:34.337103 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5a239-48ff-478d-a202-b5d03e510f4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:34.338649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5a239-48ff-478d-a202-b5d03e510f4f None admin] RBAC: Adding query filter params (group.id=99468c055e7d412b907043393618a43f, role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=a01a22dec10b499785fdfbdf0a6fcb47) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:34.339307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5a239-48ff-478d-a202-b5d03e510f4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:34.340949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5a239-48ff-478d-a202-b5d03e510f4f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:34.348635 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 193/384] 10.222.0.121 () {56 vars in 1381 bytes} [Thu May 26 17:39:34 2022] GET /identity/v3/role_assignments?group.id=99468c055e7d412b907043393618a43f&scope.project.id=a01a22dec10b499785fdfbdf0a6fcb47&role.id=4f574961798246a4b12ad2276e67685d => generated 261 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:34.350391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a10dc3-9021-4e93-931a-2d97db8e837e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:34.350919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a10dc3-9021-4e93-931a-2d97db8e837e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:34.351428 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a10dc3-9021-4e93-931a-2d97db8e837e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:34.353428 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 194/385] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:34 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:34.359706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0323c0e-c363-4625-b9f4-a9b66813b969 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:34.360318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0323c0e-c363-4625-b9f4-a9b66813b969 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:34.360923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0323c0e-c363-4625-b9f4-a9b66813b969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:34.381987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d0323c0e-c363-4625-b9f4-a9b66813b969 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:34.383754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d0323c0e-c363-4625-b9f4-a9b66813b969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:34.463364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf96e6-ae0b-41db-9746-5d7740dedc07 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:34.490299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 195/386] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:34.512457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0323c0e-c363-4625-b9f4-a9b66813b969 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:34.527638 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:34.545706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4f519-72ee-4588-8d86-db4f3653112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:34.545706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4f519-72ee-4588-8d86-db4f3653112c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:34.545706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4f519-72ee-4588-8d86-db4f3653112c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:34.545706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4f519-72ee-4588-8d86-db4f3653112c None admin] PATH_INFO: `/v3/groups/3bb70362290945a6bcba24363af6df8b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:34.548654 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4f519-72ee-4588-8d86-db4f3653112c None admin] RBAC: Authorizing `identity:get_group(group_id=3bb70362290945a6bcba24363af6df8b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:34.548654 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4f519-72ee-4588-8d86-db4f3653112c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:34.549653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 192/387] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:34 2022] GET /identity/v3/groups/3bb70362290945a6bcba24363af6df8b => generated 227 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:34.556227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4f519-72ee-4588-8d86-db4f3653112c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:34.630549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0323c0e-c363-4625-b9f4-a9b66813b969 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:34.638030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:34.673908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89674f-96a7-48b9-b1f8-4be80add9d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:34.674754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89674f-96a7-48b9-b1f8-4be80add9d4f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:34.675238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89674f-96a7-48b9-b1f8-4be80add9d4f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:34.675517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89674f-96a7-48b9-b1f8-4be80add9d4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:34.701528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89674f-96a7-48b9-b1f8-4be80add9d4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-1d89674f-96a7-48b9-b1f8-4be80add9d4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:34.710438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:34.711559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:34.712385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:34.712385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:34.712385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:34.712385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:34.712385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:34.712385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:34.712385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:34.712385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:34.712385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:34.712385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:34.712385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:34.712385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:34.714601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 196/388] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:39:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:34.724865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89674f-96a7-48b9-b1f8-4be80add9d4f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:34.758432 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4f519-72ee-4588-8d86-db4f3653112c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:34.835674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c884ef-6d48-411e-b330-bd1ec5414cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:34.836490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c884ef-6d48-411e-b330-bd1ec5414cec None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:34.836781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c884ef-6d48-411e-b330-bd1ec5414cec None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:34.837054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c884ef-6d48-411e-b330-bd1ec5414cec None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:34.838328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c884ef-6d48-411e-b330-bd1ec5414cec None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:34.840020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c884ef-6d48-411e-b330-bd1ec5414cec None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:34.842532 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 193/389] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:34 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:34.848072 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c884ef-6d48-411e-b330-bd1ec5414cec None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:34.852200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:34.884965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89674f-96a7-48b9-b1f8-4be80add9d4f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:34.922641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6bd3b-874d-4985-bb6b-ab1e8ea69427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:34.923352 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6bd3b-874d-4985-bb6b-ab1e8ea69427 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:34.923630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6bd3b-874d-4985-bb6b-ab1e8ea69427 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:34.923905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6bd3b-874d-4985-bb6b-ab1e8ea69427 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:34.924658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6bd3b-874d-4985-bb6b-ab1e8ea69427 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:34.924964 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6bd3b-874d-4985-bb6b-ab1e8ea69427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:34.927160 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6bd3b-874d-4985-bb6b-ab1e8ea69427 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:34.936621 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 197/390] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:34 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:34.942926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6bd3b-874d-4985-bb6b-ab1e8ea69427 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:35.048541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6bd3b-874d-4985-bb6b-ab1e8ea69427 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:35.075775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c884ef-6d48-411e-b330-bd1ec5414cec None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:35.083311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:35.087126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d16bf2-15e6-43a4-88a4-6c2fc9904c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:35.087947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d16bf2-15e6-43a4-88a4-6c2fc9904c99 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:35.088227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d16bf2-15e6-43a4-88a4-6c2fc9904c99 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:35.088503 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d16bf2-15e6-43a4-88a4-6c2fc9904c99 None admin] PATH_INFO: `/v3/groups/3bb70362290945a6bcba24363af6df8b` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:35.094847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d16bf2-15e6-43a4-88a4-6c2fc9904c99 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb70362290945a6bcba24363af6df8b)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:35.094847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d16bf2-15e6-43a4-88a4-6c2fc9904c99 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:35.094847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 198/391] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:34 2022] GET /identity/v3/groups/3bb70362290945a6bcba24363af6df8b => generated 227 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:35.099015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d16bf2-15e6-43a4-88a4-6c2fc9904c99 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:35.126851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:35.152403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6753700-7b00-4437-b1ba-e593c9f3915c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:35.153828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6753700-7b00-4437-b1ba-e593c9f3915c None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:35.154315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6753700-7b00-4437-b1ba-e593c9f3915c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:35.154854 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6753700-7b00-4437-b1ba-e593c9f3915c None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce/groups/3bb70362290945a6bcba24363af6df8b/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:35.157269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6753700-7b00-4437-b1ba-e593c9f3915c None admin] RBAC: Authorizing `identity:create_grant(project_id=8611e940b0fa4f8d910441467b31bdce, group_id=3bb70362290945a6bcba24363af6df8b, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:35.164409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6753700-7b00-4437-b1ba-e593c9f3915c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:35.212446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d16bf2-15e6-43a4-88a4-6c2fc9904c99 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:35.247992 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c76a9-d12f-4f98-9ca9-4a11e368a6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:35.248863 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c76a9-d12f-4f98-9ca9-4a11e368a6d2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:35.249145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c76a9-d12f-4f98-9ca9-4a11e368a6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:35.249417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c76a9-d12f-4f98-9ca9-4a11e368a6d2 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:35.250657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c76a9-d12f-4f98-9ca9-4a11e368a6d2 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:35.252095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c76a9-d12f-4f98-9ca9-4a11e368a6d2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:35.254749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 199/392] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:35 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:35.260278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c76a9-d12f-4f98-9ca9-4a11e368a6d2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:35.279715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 194/393] 10.222.0.121 () {58 vars in 1447 bytes} [Thu May 26 17:39:34 2022] PUT /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce/groups/3bb70362290945a6bcba24363af6df8b/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:39:35.334462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:35.379806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c76a9-d12f-4f98-9ca9-4a11e368a6d2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:35.445595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b61a67-e50f-46eb-a59d-78d838492726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:35.447183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b61a67-e50f-46eb-a59d-78d838492726 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:35.447675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b61a67-e50f-46eb-a59d-78d838492726 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:35.448148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b61a67-e50f-46eb-a59d-78d838492726 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce/groups/3bb70362290945a6bcba24363af6df8b/roles/582f59fc05ce474287c6fdc503f9ac82` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:35.458703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b61a67-e50f-46eb-a59d-78d838492726 None admin] RBAC: Authorizing `identity:create_grant(project_id=8611e940b0fa4f8d910441467b31bdce, group_id=3bb70362290945a6bcba24363af6df8b, role_id=582f59fc05ce474287c6fdc503f9ac82)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:35.461388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b61a67-e50f-46eb-a59d-78d838492726 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:35.581096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 200/394] 10.222.0.121 () {58 vars in 1447 bytes} [Thu May 26 17:39:35 2022] PUT /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce/groups/3bb70362290945a6bcba24363af6df8b/roles/582f59fc05ce474287c6fdc503f9ac82 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:39:36.118099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c5440-d333-4679-9bf7-c4573d394ab4 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:36.119097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c5440-d333-4679-9bf7-c4573d394ab4 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:36.119583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c5440-d333-4679-9bf7-c4573d394ab4 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:36.121190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 195/395] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:36 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:36.126943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec4571-75fd-48c4-816c-919d92022336 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:36.127398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec4571-75fd-48c4-816c-919d92022336 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:36.127619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec4571-75fd-48c4-816c-919d92022336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:36.155838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-53ec4571-75fd-48c4-816c-919d92022336 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:36.157034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-53ec4571-75fd-48c4-816c-919d92022336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:36.245473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 201/396] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:36.280708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c5440-d333-4679-9bf7-c4573d394ab4 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:36.430296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c5440-d333-4679-9bf7-c4573d394ab4 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:36.469219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbe513-b098-47b7-99cc-b46e70c72f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:36.470016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbe513-b098-47b7-99cc-b46e70c72f48 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:36.470298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbe513-b098-47b7-99cc-b46e70c72f48 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:36.470567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbe513-b098-47b7-99cc-b46e70c72f48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:36.483611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbe513-b098-47b7-99cc-b46e70c72f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-4ecbe513-b098-47b7-99cc-b46e70c72f48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:36.492474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:36.493420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:36.494197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 196/397] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:36.499693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec4571-75fd-48c4-816c-919d92022336 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:36.587917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:36.640521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec4571-75fd-48c4-816c-919d92022336 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:36.686989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4401493-829f-4e80-be22-abcecc9d6457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:36.688157 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4401493-829f-4e80-be22-abcecc9d6457 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:36.688434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4401493-829f-4e80-be22-abcecc9d6457 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:36.688707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4401493-829f-4e80-be22-abcecc9d6457 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:36.689474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4401493-829f-4e80-be22-abcecc9d6457 None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:36.689762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4401493-829f-4e80-be22-abcecc9d6457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:36.691219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4401493-829f-4e80-be22-abcecc9d6457 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:36.705064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 202/398] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:36 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:36.710948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbe513-b098-47b7-99cc-b46e70c72f48 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:36.797248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:36.890950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbe513-b098-47b7-99cc-b46e70c72f48 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:36.933137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d83d3-3010-4a5c-bf75-e1adbbf67dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:36.934241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d83d3-3010-4a5c-bf75-e1adbbf67dd6 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:36.934518 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d83d3-3010-4a5c-bf75-e1adbbf67dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:36.934786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d83d3-3010-4a5c-bf75-e1adbbf67dd6 None admin] PATH_INFO: `/v3/groups/3bb70362290945a6bcba24363af6df8b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:36.937915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d83d3-3010-4a5c-bf75-e1adbbf67dd6 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb70362290945a6bcba24363af6df8b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:36.939336 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d83d3-3010-4a5c-bf75-e1adbbf67dd6 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:36.941343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 197/399] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:36 2022] GET /identity/v3/groups/3bb70362290945a6bcba24363af6df8b => generated 227 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:36.947148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4401493-829f-4e80-be22-abcecc9d6457 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:37.071200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4401493-829f-4e80-be22-abcecc9d6457 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:37.093901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d49c4-7faf-4bad-a441-f442e251ec57 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:37.093901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d49c4-7faf-4bad-a441-f442e251ec57 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:37.093901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d49c4-7faf-4bad-a441-f442e251ec57 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:37.093901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 198/400] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:37.097872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:37.106422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099bf70-8ba8-4a29-9773-f92e8646457e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:37.106698 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099bf70-8ba8-4a29-9773-f92e8646457e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:37.107000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099bf70-8ba8-4a29-9773-f92e8646457e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:37.124383 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad539cba-3c7f-4d05-9b9f-fce3a3614421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:37.125656 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad539cba-3c7f-4d05-9b9f-fce3a3614421 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:37.126145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad539cba-3c7f-4d05-9b9f-fce3a3614421 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:37.126623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad539cba-3c7f-4d05-9b9f-fce3a3614421 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:37.128363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad539cba-3c7f-4d05-9b9f-fce3a3614421 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:37.131343 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad539cba-3c7f-4d05-9b9f-fce3a3614421 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:37.133677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 203/401] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:36 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:37.143911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad539cba-3c7f-4d05-9b9f-fce3a3614421 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:37.155128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5099bf70-8ba8-4a29-9773-f92e8646457e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:37.157480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5099bf70-8ba8-4a29-9773-f92e8646457e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:37.256255 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 199/402] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:37.287509 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad539cba-3c7f-4d05-9b9f-fce3a3614421 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:37.291497 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099bf70-8ba8-4a29-9773-f92e8646457e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:37.337632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:37.339630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91998fda-447e-49e0-a87e-ed3c302fa7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:37.340305 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91998fda-447e-49e0-a87e-ed3c302fa7f8 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:37.340588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91998fda-447e-49e0-a87e-ed3c302fa7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:37.340861 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91998fda-447e-49e0-a87e-ed3c302fa7f8 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62/groups/3bb70362290945a6bcba24363af6df8b/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:37.348919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91998fda-447e-49e0-a87e-ed3c302fa7f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=77149f8497f3433c8a1c236067c9eb62, group_id=3bb70362290945a6bcba24363af6df8b, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:37.353166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91998fda-447e-49e0-a87e-ed3c302fa7f8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:37.424462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099bf70-8ba8-4a29-9773-f92e8646457e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:37.464118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f7732-2abf-4094-82c8-f4530cf2f6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:37.464998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f7732-2abf-4094-82c8-f4530cf2f6ac None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:37.465249 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f7732-2abf-4094-82c8-f4530cf2f6ac None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:37.465680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f7732-2abf-4094-82c8-f4530cf2f6ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:37.477974 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 204/403] 10.222.0.121 () {58 vars in 1447 bytes} [Thu May 26 17:39:37 2022] PUT /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62/groups/3bb70362290945a6bcba24363af6df8b/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:39:37.479646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f35aab-813f-46e6-bb04-9d90cb59124c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:37.480147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f35aab-813f-46e6-bb04-9d90cb59124c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:37.480625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f35aab-813f-46e6-bb04-9d90cb59124c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:37.480752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f7732-2abf-4094-82c8-f4530cf2f6ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:37.482617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 205/404] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:37.488742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd22cb-3c1e-4035-b488-dfe012ca2f8e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:37.489364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd22cb-3c1e-4035-b488-dfe012ca2f8e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-561f7732-2abf-4094-82c8-f4530cf2f6ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:37.489841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:37.490716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:37.491553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:37.491553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:37.491553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:37.491553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:37.491553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:37.491553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:37.491553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:37.491553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:37.491553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:37.491553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:37.491553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:37.491553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:37.492146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd22cb-3c1e-4035-b488-dfe012ca2f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:37.492215 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 200/405] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:39:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:37.498201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f7732-2abf-4094-82c8-f4530cf2f6ac None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:37.527637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8cfd22cb-3c1e-4035-b488-dfe012ca2f8e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:37.529164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8cfd22cb-3c1e-4035-b488-dfe012ca2f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:37.592379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:37.627511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 206/406] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:37.650613 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd22cb-3c1e-4035-b488-dfe012ca2f8e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:37.715471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:37.720100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f7732-2abf-4094-82c8-f4530cf2f6ac None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:37.762693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7763d89-33f2-4b52-aa1e-5854659d79c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:37.763788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7763d89-33f2-4b52-aa1e-5854659d79c8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:37.764216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7763d89-33f2-4b52-aa1e-5854659d79c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:37.764635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7763d89-33f2-4b52-aa1e-5854659d79c8 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:37.765654 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7763d89-33f2-4b52-aa1e-5854659d79c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:37.766102 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7763d89-33f2-4b52-aa1e-5854659d79c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:37.768152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7763d89-33f2-4b52-aa1e-5854659d79c8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:37.783327 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 201/407] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:37 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:37.789275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7763d89-33f2-4b52-aa1e-5854659d79c8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:37.812128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd22cb-3c1e-4035-b488-dfe012ca2f8e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:37.850899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e9e7e-91d4-4cc0-9fea-3f4a8a1b0b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:37.851768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e9e7e-91d4-4cc0-9fea-3f4a8a1b0b3f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:37.852046 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e9e7e-91d4-4cc0-9fea-3f4a8a1b0b3f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:37.852316 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e9e7e-91d4-4cc0-9fea-3f4a8a1b0b3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:37.864462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e9e7e-91d4-4cc0-9fea-3f4a8a1b0b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-fe4e9e7e-91d4-4cc0-9fea-3f4a8a1b0b3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:37.873470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:37.874588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:37.875513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 207/408] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:39:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:37.881785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e9e7e-91d4-4cc0-9fea-3f4a8a1b0b3f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:37.928377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7763d89-33f2-4b52-aa1e-5854659d79c8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:37.983054 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:38.012750 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f71413-457e-4049-9e1f-bc79b70a92e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:38.013813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f71413-457e-4049-9e1f-bc79b70a92e0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:38.014186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f71413-457e-4049-9e1f-bc79b70a92e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:38.014567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f71413-457e-4049-9e1f-bc79b70a92e0 None admin] PATH_INFO: `/v3/groups/3bb70362290945a6bcba24363af6df8b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:38.019615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f71413-457e-4049-9e1f-bc79b70a92e0 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb70362290945a6bcba24363af6df8b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:38.021072 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f71413-457e-4049-9e1f-bc79b70a92e0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:38.022768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 202/409] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:37 2022] GET /identity/v3/groups/3bb70362290945a6bcba24363af6df8b => generated 227 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:38.031990 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f71413-457e-4049-9e1f-bc79b70a92e0 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:38.075292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:38.115022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e9e7e-91d4-4cc0-9fea-3f4a8a1b0b3f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:38.184520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f71413-457e-4049-9e1f-bc79b70a92e0 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:38.204150 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86feee9e-09c9-44d0-abd9-7251b2ae84e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:38.204841 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86feee9e-09c9-44d0-abd9-7251b2ae84e5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:38.205695 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86feee9e-09c9-44d0-abd9-7251b2ae84e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:38.205976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86feee9e-09c9-44d0-abd9-7251b2ae84e5 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:38.207078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86feee9e-09c9-44d0-abd9-7251b2ae84e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:38.207388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86feee9e-09c9-44d0-abd9-7251b2ae84e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:38.209487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86feee9e-09c9-44d0-abd9-7251b2ae84e5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:38.227696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 208/410] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:37 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:38.229658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77835902-57f3-488d-8178-711df7c29d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:38.230600 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77835902-57f3-488d-8178-711df7c29d4f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:38.231041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77835902-57f3-488d-8178-711df7c29d4f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:38.231366 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77835902-57f3-488d-8178-711df7c29d4f None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:38.232667 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77835902-57f3-488d-8178-711df7c29d4f None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:38.234149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77835902-57f3-488d-8178-711df7c29d4f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:38.235875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86feee9e-09c9-44d0-abd9-7251b2ae84e5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:38.241910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 203/411] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:38 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:38.252733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77835902-57f3-488d-8178-711df7c29d4f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:38.429466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:38.437187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77835902-57f3-488d-8178-711df7c29d4f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:38.470800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:38.474048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86feee9e-09c9-44d0-abd9-7251b2ae84e5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:38.477668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8e328-e70e-4dbe-a8a6-f0315b8baa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:38.478859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8e328-e70e-4dbe-a8a6-f0315b8baa7e None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:38.479329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8e328-e70e-4dbe-a8a6-f0315b8baa7e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:38.479824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8e328-e70e-4dbe-a8a6-f0315b8baa7e None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62/groups/3bb70362290945a6bcba24363af6df8b/roles/582f59fc05ce474287c6fdc503f9ac82` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:38.491664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8e328-e70e-4dbe-a8a6-f0315b8baa7e None admin] RBAC: Authorizing `identity:create_grant(project_id=77149f8497f3433c8a1c236067c9eb62, group_id=3bb70362290945a6bcba24363af6df8b, role_id=582f59fc05ce474287c6fdc503f9ac82)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:38.491664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8e328-e70e-4dbe-a8a6-f0315b8baa7e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:38.510416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fb44b-1152-42d7-9aee-2edab43fb353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:38.511504 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fb44b-1152-42d7-9aee-2edab43fb353 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:38.515246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fb44b-1152-42d7-9aee-2edab43fb353 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:38.515246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fb44b-1152-42d7-9aee-2edab43fb353 None admin] PATH_INFO: `/v3/groups/99468c055e7d412b907043393618a43f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:38.515246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fb44b-1152-42d7-9aee-2edab43fb353 None admin] RBAC: Authorizing `identity:get_group(group_id=99468c055e7d412b907043393618a43f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:38.515246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fb44b-1152-42d7-9aee-2edab43fb353 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:38.516807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 209/412] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:38 2022] GET /identity/v3/groups/99468c055e7d412b907043393618a43f => generated 230 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:38.517971 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddb8b9-87e7-4fbe-bd48-1d514c4d9394 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:38.518255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddb8b9-87e7-4fbe-bd48-1d514c4d9394 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:38.518528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddb8b9-87e7-4fbe-bd48-1d514c4d9394 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:38.524306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 210/413] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:38.524306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196720f-cc3d-4025-ace2-7f0c62bb9776 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:38.524306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196720f-cc3d-4025-ace2-7f0c62bb9776 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:38.524306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196720f-cc3d-4025-ace2-7f0c62bb9776 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:38.524306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 211/414] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:38.524692 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196720f-cc3d-4025-ace2-7f0c62bb9776 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:38.589837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 204/415] 10.222.0.121 () {58 vars in 1447 bytes} [Thu May 26 17:39:38 2022] PUT /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62/groups/3bb70362290945a6bcba24363af6df8b/roles/582f59fc05ce474287c6fdc503f9ac82 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:39:38.589837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cae02-c1df-495c-8459-d21f9fa67a03 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:38.589837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cae02-c1df-495c-8459-d21f9fa67a03 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:38.589837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cae02-c1df-495c-8459-d21f9fa67a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:38.605114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-2e1cae02-c1df-495c-8459-d21f9fa67a03 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:38.606406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-2e1cae02-c1df-495c-8459-d21f9fa67a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:38.654194 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196720f-cc3d-4025-ace2-7f0c62bb9776 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:38.698111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b937d32-d72f-4677-825c-6b4beedb803a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:38.699196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b937d32-d72f-4677-825c-6b4beedb803a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:38.699489 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b937d32-d72f-4677-825c-6b4beedb803a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:38.699771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b937d32-d72f-4677-825c-6b4beedb803a None admin] PATH_INFO: `/v3/projects/a01a22dec10b499785fdfbdf0a6fcb47` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:38.701429 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b937d32-d72f-4677-825c-6b4beedb803a None admin] RBAC: Authorizing `identity:get_project(project_id=a01a22dec10b499785fdfbdf0a6fcb47)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:38.702643 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 205/416] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:38.703099 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b937d32-d72f-4677-825c-6b4beedb803a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:38.704103 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2e361-69bf-417f-95f2-3467dcf1abaf None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:38.704440 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2e361-69bf-417f-95f2-3467dcf1abaf None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:38.704723 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2e361-69bf-417f-95f2-3467dcf1abaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:38.705114 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 212/417] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:38 2022] GET /identity/v3/projects/a01a22dec10b499785fdfbdf0a6fcb47 => generated 345 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:38.713323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b937d32-d72f-4677-825c-6b4beedb803a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:38.733074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3cb2e361-69bf-417f-95f2-3467dcf1abaf None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:38.734293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3cb2e361-69bf-417f-95f2-3467dcf1abaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:38.771026 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:38.823498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 206/418] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:38.824686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2e361-69bf-417f-95f2-3467dcf1abaf None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:38.846411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:38.865197 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b937d32-d72f-4677-825c-6b4beedb803a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:38.916426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a848ab3-d51e-4301-bea2-4ebd509b1131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:38.918436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a848ab3-d51e-4301-bea2-4ebd509b1131 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:38.918967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a848ab3-d51e-4301-bea2-4ebd509b1131 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:38.919888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a848ab3-d51e-4301-bea2-4ebd509b1131 None admin] PATH_INFO: `/v3/projects/a01a22dec10b499785fdfbdf0a6fcb47/groups/99468c055e7d412b907043393618a43f/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:38.922756 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a848ab3-d51e-4301-bea2-4ebd509b1131 None admin] RBAC: Authorizing `identity:create_grant(project_id=a01a22dec10b499785fdfbdf0a6fcb47, group_id=99468c055e7d412b907043393618a43f, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:38.925206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a848ab3-d51e-4301-bea2-4ebd509b1131 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:38.946328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2e361-69bf-417f-95f2-3467dcf1abaf None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:38.983930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446bf3a-5bae-47e6-991b-f8badfaf452f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:38.984656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446bf3a-5bae-47e6-991b-f8badfaf452f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:38.984943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446bf3a-5bae-47e6-991b-f8badfaf452f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:38.985216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446bf3a-5bae-47e6-991b-f8badfaf452f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:38.997450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446bf3a-5bae-47e6-991b-f8badfaf452f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:39.001593 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 213/419] 10.222.0.121 () {58 vars in 1447 bytes} [Thu May 26 17:39:38 2022] PUT /identity/v3/projects/a01a22dec10b499785fdfbdf0a6fcb47/groups/99468c055e7d412b907043393618a43f/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:39:39.002692 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a848ab3-d51e-4301-bea2-4ebd509b1131 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-a446bf3a-5bae-47e6-991b-f8badfaf452f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:39.005806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:39.006761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:39.008345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 207/420] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:39.013567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446bf3a-5bae-47e6-991b-f8badfaf452f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:39.105880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:39.137029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446bf3a-5bae-47e6-991b-f8badfaf452f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:39.145423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:39.155061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a848ab3-d51e-4301-bea2-4ebd509b1131 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:39.201996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395edad-efa9-4e7f-bf62-37e6425992a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:39.203603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395edad-efa9-4e7f-bf62-37e6425992a7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:39.204130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395edad-efa9-4e7f-bf62-37e6425992a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:39.205134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395edad-efa9-4e7f-bf62-37e6425992a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:39.221404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395edad-efa9-4e7f-bf62-37e6425992a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:39.228290 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a1743-810d-4c9b-8eeb-04f5f676beeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:39.228979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a1743-810d-4c9b-8eeb-04f5f676beeb None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:39.229266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a1743-810d-4c9b-8eeb-04f5f676beeb None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:39.229602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a1743-810d-4c9b-8eeb-04f5f676beeb None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:39.230397 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a1743-810d-4c9b-8eeb-04f5f676beeb None admin] RBAC: Adding query filter params (name=member) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:39.230705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a1743-810d-4c9b-8eeb-04f5f676beeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:39.232183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a1743-810d-4c9b-8eeb-04f5f676beeb None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-4395edad-efa9-4e7f-bf62-37e6425992a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:39.233036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:39.233925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:39.234854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:39.234854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:39.234854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:39.234854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:39.234854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:39.234854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:39.234854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:39.234854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:39.234854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:39.234854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:39.234854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:39.234854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:39.237670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 214/421] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:39:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:39.245250 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395edad-efa9-4e7f-bf62-37e6425992a7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:39.257220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 208/422] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:39 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:39.262413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a1743-810d-4c9b-8eeb-04f5f676beeb None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:39.401881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395edad-efa9-4e7f-bf62-37e6425992a7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:39.421022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a1743-810d-4c9b-8eeb-04f5f676beeb None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:39.446809 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395194c-b610-4654-af9c-b06b9dd1aaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:39.447863 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395194c-b610-4654-af9c-b06b9dd1aaca None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:39.448331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395194c-b610-4654-af9c-b06b9dd1aaca None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:39.448773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395194c-b610-4654-af9c-b06b9dd1aaca None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:39.449686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395194c-b610-4654-af9c-b06b9dd1aaca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:39.450172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395194c-b610-4654-af9c-b06b9dd1aaca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:39.451883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395194c-b610-4654-af9c-b06b9dd1aaca None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:39.464084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 215/423] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:39 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:39.474122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395194c-b610-4654-af9c-b06b9dd1aaca None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:39.503294 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f5bf0-8c71-481e-9e50-5f823cac8502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:39.504212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f5bf0-8c71-481e-9e50-5f823cac8502 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:39.504640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f5bf0-8c71-481e-9e50-5f823cac8502 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:39.504920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f5bf0-8c71-481e-9e50-5f823cac8502 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:39.510468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:39.511748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:39.634229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f5bf0-8c71-481e-9e50-5f823cac8502 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:39.635732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f5bf0-8c71-481e-9e50-5f823cac8502 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:39.674357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395194c-b610-4654-af9c-b06b9dd1aaca None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:39.736221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68645a2-d73f-40e5-bcfe-cc3cb2875ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:39.736885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68645a2-d73f-40e5-bcfe-cc3cb2875ca1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:39.751011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68645a2-d73f-40e5-bcfe-cc3cb2875ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:39.751011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68645a2-d73f-40e5-bcfe-cc3cb2875ca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:39.833190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68645a2-d73f-40e5-bcfe-cc3cb2875ca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:39.834593 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68645a2-d73f-40e5-bcfe-cc3cb2875ca1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:39.848725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 209/424] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:39 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:39:39.870432 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f5bf0-8c71-481e-9e50-5f823cac8502 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:39.871023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:39.917723 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:40.015612 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 216/425] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:39 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:39:40.016875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68645a2-d73f-40e5-bcfe-cc3cb2875ca1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:40.044241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f5bf0-8c71-481e-9e50-5f823cac8502 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:40.083739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345feb27-1910-4766-988f-f34b738e1899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:40.084642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345feb27-1910-4766-988f-f34b738e1899 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:40.085007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345feb27-1910-4766-988f-f34b738e1899 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:40.085350 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345feb27-1910-4766-988f-f34b738e1899 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:40.086708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345feb27-1910-4766-988f-f34b738e1899 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:40.088304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345feb27-1910-4766-988f-f34b738e1899 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:40.090441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 210/426] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:39 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:40.091730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730f390-f1e2-4b9f-9b3c-ff01b43c45e2 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:40.092010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730f390-f1e2-4b9f-9b3c-ff01b43c45e2 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:40.092279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730f390-f1e2-4b9f-9b3c-ff01b43c45e2 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:40.093938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 211/427] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:40.098715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730f390-f1e2-4b9f-9b3c-ff01b43c45e2 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:40.171847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68645a2-d73f-40e5-bcfe-cc3cb2875ca1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:40.192068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:40.211590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730f390-f1e2-4b9f-9b3c-ff01b43c45e2 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:40.211979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed7039-90f8-4b77-9261-4441c428fd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:40.212963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed7039-90f8-4b77-9261-4441c428fd12 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:40.213345 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed7039-90f8-4b77-9261-4441c428fd12 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:40.213726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed7039-90f8-4b77-9261-4441c428fd12 None admin] PATH_INFO: `/v3/projects/8611e940b0fa4f8d910441467b31bdce` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:40.215518 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:40.216250 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed7039-90f8-4b77-9261-4441c428fd12 None admin] RBAC: Authorizing `identity:get_project(project_id=8611e940b0fa4f8d910441467b31bdce)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:40.218103 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed7039-90f8-4b77-9261-4441c428fd12 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:40.220485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 217/428] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:40 2022] GET /identity/v3/projects/8611e940b0fa4f8d910441467b31bdce => generated 299 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:40.222534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965aa52-5774-4162-b749-2b221aedd7f3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:40.223869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965aa52-5774-4162-b749-2b221aedd7f3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:40.224405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965aa52-5774-4162-b749-2b221aedd7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:40.242772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4965aa52-5774-4162-b749-2b221aedd7f3 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:40.244347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4965aa52-5774-4162-b749-2b221aedd7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:40.251097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753cf5-a4ef-4502-8d18-8cf86b902773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:40.251963 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753cf5-a4ef-4502-8d18-8cf86b902773 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:40.252238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753cf5-a4ef-4502-8d18-8cf86b902773 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:40.252507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753cf5-a4ef-4502-8d18-8cf86b902773 None admin] PATH_INFO: `/v3/groups/3bb70362290945a6bcba24363af6df8b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:40.254281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753cf5-a4ef-4502-8d18-8cf86b902773 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb70362290945a6bcba24363af6df8b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:40.255902 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753cf5-a4ef-4502-8d18-8cf86b902773 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:40.263303 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 212/429] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:40 2022] GET /identity/v3/groups/3bb70362290945a6bcba24363af6df8b => generated 227 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:40.263303 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753cf5-a4ef-4502-8d18-8cf86b902773 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:40.327945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 218/430] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:40.329180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965aa52-5774-4162-b749-2b221aedd7f3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:40.368878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753cf5-a4ef-4502-8d18-8cf86b902773 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:40.407602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edea3f-e45c-4845-98c9-8d1fe48cfd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:40.408088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edea3f-e45c-4845-98c9-8d1fe48cfd02 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:40.408463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edea3f-e45c-4845-98c9-8d1fe48cfd02 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:40.408832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edea3f-e45c-4845-98c9-8d1fe48cfd02 None admin] PATH_INFO: `/v3/groups/3bb70362290945a6bcba24363af6df8b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:40.411000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86edea3f-e45c-4845-98c9-8d1fe48cfd02 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb70362290945a6bcba24363af6df8b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:40.412530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86edea3f-e45c-4845-98c9-8d1fe48cfd02 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:40.414666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 213/431] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:40 2022] GET /identity/v3/groups/3bb70362290945a6bcba24363af6df8b => generated 227 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:40.415382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edea3f-e45c-4845-98c9-8d1fe48cfd02 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:40.443192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965aa52-5774-4162-b749-2b221aedd7f3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:40.462127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:40.464615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:40.484483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeacc02-cee1-41f1-bb0f-f01c56652949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:40.485418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeacc02-cee1-41f1-bb0f-f01c56652949 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:40.485848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeacc02-cee1-41f1-bb0f-f01c56652949 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:40.486312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeacc02-cee1-41f1-bb0f-f01c56652949 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:40.487372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeacc02-cee1-41f1-bb0f-f01c56652949 None admin] RBAC: Adding query filter params (group.id=3bb70362290945a6bcba24363af6df8b, role.id=e87c07d8d70d46d89a2c4f7246edeff6, scope.project.id=8611e940b0fa4f8d910441467b31bdce) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:40.488488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeacc02-cee1-41f1-bb0f-f01c56652949 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:40.490259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeacc02-cee1-41f1-bb0f-f01c56652949 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:40.498016 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 219/432] 10.222.0.121 () {56 vars in 1381 bytes} [Thu May 26 17:39:40 2022] GET /identity/v3/role_assignments?group.id=3bb70362290945a6bcba24363af6df8b&scope.project.id=8611e940b0fa4f8d910441467b31bdce&role.id=e87c07d8d70d46d89a2c4f7246edeff6 => generated 613 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:40.499097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeacc02-cee1-41f1-bb0f-f01c56652949 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:40.560357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edea3f-e45c-4845-98c9-8d1fe48cfd02 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:40.635321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a3448-ecc8-47ed-8ff8-0e941d7f6de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:40.636051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a3448-ecc8-47ed-8ff8-0e941d7f6de1 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:40.636328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a3448-ecc8-47ed-8ff8-0e941d7f6de1 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:40.636736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a3448-ecc8-47ed-8ff8-0e941d7f6de1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:40.661156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a3448-ecc8-47ed-8ff8-0e941d7f6de1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-283a3448-ecc8-47ed-8ff8-0e941d7f6de1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:40.669391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:40.670363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:40.671462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 214/433] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:40.681291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a3448-ecc8-47ed-8ff8-0e941d7f6de1 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:40.717260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeacc02-cee1-41f1-bb0f-f01c56652949 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:40.763542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86f0c3-a666-41b0-9207-bcd1c839dc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:40.764508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86f0c3-a666-41b0-9207-bcd1c839dc8f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:40.764894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86f0c3-a666-41b0-9207-bcd1c839dc8f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:40.765245 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86f0c3-a666-41b0-9207-bcd1c839dc8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:40.766076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86f0c3-a666-41b0-9207-bcd1c839dc8f None admin] RBAC: Adding query filter params (group.id=3bb70362290945a6bcba24363af6df8b, role.id=582f59fc05ce474287c6fdc503f9ac82, scope.project.id=8611e940b0fa4f8d910441467b31bdce) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:40.766384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86f0c3-a666-41b0-9207-bcd1c839dc8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:40.769398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86f0c3-a666-41b0-9207-bcd1c839dc8f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:40.776396 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 220/434] 10.222.0.121 () {56 vars in 1381 bytes} [Thu May 26 17:39:40 2022] GET /identity/v3/role_assignments?group.id=3bb70362290945a6bcba24363af6df8b&scope.project.id=8611e940b0fa4f8d910441467b31bdce&role.id=582f59fc05ce474287c6fdc503f9ac82 => generated 613 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:40.821488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:40.824543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a3448-ecc8-47ed-8ff8-0e941d7f6de1 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:40.866596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6af0a-e343-44d1-88a8-ac9467a7e0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:40.867323 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6af0a-e343-44d1-88a8-ac9467a7e0a8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:40.867603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6af0a-e343-44d1-88a8-ac9467a7e0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:40.867875 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6af0a-e343-44d1-88a8-ac9467a7e0a8 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:40.868635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6af0a-e343-44d1-88a8-ac9467a7e0a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:40.868939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6af0a-e343-44d1-88a8-ac9467a7e0a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:40.870340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6af0a-e343-44d1-88a8-ac9467a7e0a8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:40.880209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 215/435] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:39:40 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:40.885708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86f0c3-a666-41b0-9207-bcd1c839dc8f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:40.944860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:41.011133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86f0c3-a666-41b0-9207-bcd1c839dc8f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:41.047891 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092c518-b09b-4808-bf7e-112a0c0dc3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:41.048545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092c518-b09b-4808-bf7e-112a0c0dc3a7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:41.048824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092c518-b09b-4808-bf7e-112a0c0dc3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:41.049097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092c518-b09b-4808-bf7e-112a0c0dc3a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:41.136991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092c518-b09b-4808-bf7e-112a0c0dc3a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:41.138172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092c518-b09b-4808-bf7e-112a0c0dc3a7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:41.194023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:41.262179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 221/436] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:40 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:41.267425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6af0a-e343-44d1-88a8-ac9467a7e0a8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:41.319354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dcac5-cdb4-43df-8b33-a701a392663a None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:41.319694 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dcac5-cdb4-43df-8b33-a701a392663a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:41.319910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dcac5-cdb4-43df-8b33-a701a392663a None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:41.321409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 222/437] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:41.326596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7f8a3-938f-49f1-8424-e1d7449b2ba0 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:41.326909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7f8a3-938f-49f1-8424-e1d7449b2ba0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:41.327191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7f8a3-938f-49f1-8424-e1d7449b2ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:41.349471 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-96c7f8a3-938f-49f1-8424-e1d7449b2ba0 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:41.351135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-96c7f8a3-938f-49f1-8424-e1d7449b2ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:41.399083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6af0a-e343-44d1-88a8-ac9467a7e0a8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:41.440490 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 223/438] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:41.446897 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993568d-1e44-4e21-b493-c616d4d74404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:41.448029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993568d-1e44-4e21-b493-c616d4d74404 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:41.448482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993568d-1e44-4e21-b493-c616d4d74404 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:41.448923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993568d-1e44-4e21-b493-c616d4d74404 None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:41.450388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993568d-1e44-4e21-b493-c616d4d74404 None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:41.452049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993568d-1e44-4e21-b493-c616d4d74404 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:41.453906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 216/439] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:41 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:41.460100 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7f8a3-938f-49f1-8424-e1d7449b2ba0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:41.464977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993568d-1e44-4e21-b493-c616d4d74404 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:41.505442 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:41.549360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:41.583038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7f8a3-938f-49f1-8424-e1d7449b2ba0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:41.590399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993568d-1e44-4e21-b493-c616d4d74404 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:41.625744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d814b20-f4f2-437e-aa9c-b1465046c1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:41.626603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d814b20-f4f2-437e-aa9c-b1465046c1b9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:41.626912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d814b20-f4f2-437e-aa9c-b1465046c1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:41.627189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d814b20-f4f2-437e-aa9c-b1465046c1b9 None admin] PATH_INFO: `/v3/groups/3bb70362290945a6bcba24363af6df8b` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:41.628457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d814b20-f4f2-437e-aa9c-b1465046c1b9 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb70362290945a6bcba24363af6df8b)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:41.629539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271606d-4503-4766-9da1-689cb439cb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:41.630008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d814b20-f4f2-437e-aa9c-b1465046c1b9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:41.630346 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271606d-4503-4766-9da1-689cb439cb8d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:41.630625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271606d-4503-4766-9da1-689cb439cb8d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:41.630932 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271606d-4503-4766-9da1-689cb439cb8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:41.631667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 224/440] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:41 2022] GET /identity/v3/groups/3bb70362290945a6bcba24363af6df8b => generated 227 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:41.637420 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d814b20-f4f2-437e-aa9c-b1465046c1b9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:41.645009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271606d-4503-4766-9da1-689cb439cb8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-9271606d-4503-4766-9da1-689cb439cb8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:41.654531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:41.655450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:41.656239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 217/441] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:39:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:41.662193 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271606d-4503-4766-9da1-689cb439cb8d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:41.759932 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:41.760391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d814b20-f4f2-437e-aa9c-b1465046c1b9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:41.769933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271606d-4503-4766-9da1-689cb439cb8d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:41.795642 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b48495-37c5-4202-90da-da034cdf7a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:41.796292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b48495-37c5-4202-90da-da034cdf7a06 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:41.796568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b48495-37c5-4202-90da-da034cdf7a06 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:41.796841 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b48495-37c5-4202-90da-da034cdf7a06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:41.797656 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b48495-37c5-4202-90da-da034cdf7a06 None admin] RBAC: Adding query filter params (group.id=3bb70362290945a6bcba24363af6df8b, role.id=e87c07d8d70d46d89a2c4f7246edeff6, scope.project.id=77149f8497f3433c8a1c236067c9eb62) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:41.797965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b48495-37c5-4202-90da-da034cdf7a06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:41.799446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b48495-37c5-4202-90da-da034cdf7a06 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:41.805831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cc4c8-6741-4015-88ca-90b76c825ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:41.808164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 225/442] 10.222.0.121 () {56 vars in 1381 bytes} [Thu May 26 17:39:41 2022] GET /identity/v3/role_assignments?group.id=3bb70362290945a6bcba24363af6df8b&scope.project.id=77149f8497f3433c8a1c236067c9eb62&role.id=e87c07d8d70d46d89a2c4f7246edeff6 => generated 613 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:41.808269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cc4c8-6741-4015-88ca-90b76c825ace None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:41.808269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cc4c8-6741-4015-88ca-90b76c825ace None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:41.808269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cc4c8-6741-4015-88ca-90b76c825ace None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:41.808457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164a916-d4dd-4b30-8402-ddbaf09a4914 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:41.808457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164a916-d4dd-4b30-8402-ddbaf09a4914 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:41.808594 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cc4c8-6741-4015-88ca-90b76c825ace None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:41.808674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164a916-d4dd-4b30-8402-ddbaf09a4914 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:41.808742 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cc4c8-6741-4015-88ca-90b76c825ace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:41.809223 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 226/443] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:41.809757 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cc4c8-6741-4015-88ca-90b76c825ace None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:41.814954 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f00db8-da85-4501-b037-70277548d4f1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:41.815234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f00db8-da85-4501-b037-70277548d4f1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:41.816439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f00db8-da85-4501-b037-70277548d4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:41.827169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 218/444] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:41 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:41.829972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:41.833587 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cc4c8-6741-4015-88ca-90b76c825ace None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:41.837418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-52f00db8-da85-4501-b037-70277548d4f1 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:41.838627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-52f00db8-da85-4501-b037-70277548d4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:41.919422 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 227/445] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:39:41.946012 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f00db8-da85-4501-b037-70277548d4f1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:41.958373 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cc4c8-6741-4015-88ca-90b76c825ace None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:41.996752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d3ceb-5abb-43bc-9114-159f5baa406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:41.997410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d3ceb-5abb-43bc-9114-159f5baa406c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:41.997688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d3ceb-5abb-43bc-9114-159f5baa406c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:41.997957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d3ceb-5abb-43bc-9114-159f5baa406c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:42.024237 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:42.086256 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f00db8-da85-4501-b037-70277548d4f1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:42.087492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d3ceb-5abb-43bc-9114-159f5baa406c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:42.088951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d3ceb-5abb-43bc-9114-159f5baa406c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:42.131275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70e267-ed26-44f1-b7e8-2e25914c7302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:42.132083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70e267-ed26-44f1-b7e8-2e25914c7302 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:42.132370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70e267-ed26-44f1-b7e8-2e25914c7302 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:42.132668 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70e267-ed26-44f1-b7e8-2e25914c7302 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:42.136599 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:42.145905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70e267-ed26-44f1-b7e8-2e25914c7302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-ac70e267-ed26-44f1-b7e8-2e25914c7302 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:42.153646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:42.154577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:42.155464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 228/446] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:39:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:42.162060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70e267-ed26-44f1-b7e8-2e25914c7302 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:42.214934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 219/447] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:41 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:42.220844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d3ceb-5abb-43bc-9114-159f5baa406c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:42.274943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:42.280934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70e267-ed26-44f1-b7e8-2e25914c7302 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:42.315628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f025ec8-d0f5-48ba-83f8-5c9af7fb275b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:42.316327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f025ec8-d0f5-48ba-83f8-5c9af7fb275b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:42.316645 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f025ec8-d0f5-48ba-83f8-5c9af7fb275b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:42.317090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f025ec8-d0f5-48ba-83f8-5c9af7fb275b None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:42.319896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f025ec8-d0f5-48ba-83f8-5c9af7fb275b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:42.319896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f025ec8-d0f5-48ba-83f8-5c9af7fb275b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:42.319896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f025ec8-d0f5-48ba-83f8-5c9af7fb275b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:42.328443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 229/448] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:42 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:42.329177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d3ceb-5abb-43bc-9114-159f5baa406c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:42.335110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f025ec8-d0f5-48ba-83f8-5c9af7fb275b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:42.367714 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a6356-6c15-461d-8a74-38ec5bc15d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:42.368662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a6356-6c15-461d-8a74-38ec5bc15d8b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:42.368939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a6356-6c15-461d-8a74-38ec5bc15d8b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:42.369212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a6356-6c15-461d-8a74-38ec5bc15d8b None admin] PATH_INFO: `/v3/projects/77149f8497f3433c8a1c236067c9eb62` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:42.370393 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a6356-6c15-461d-8a74-38ec5bc15d8b None admin] RBAC: Authorizing `identity:get_project(project_id=77149f8497f3433c8a1c236067c9eb62)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:42.371891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a6356-6c15-461d-8a74-38ec5bc15d8b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:42.373571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 220/449] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:42 2022] GET /identity/v3/projects/77149f8497f3433c8a1c236067c9eb62 => generated 303 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:42.379039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a6356-6c15-461d-8a74-38ec5bc15d8b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:42.394852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:42.437967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f025ec8-d0f5-48ba-83f8-5c9af7fb275b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:42.472208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b9123-2254-4030-b96e-047350f0371d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:42.473198 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b9123-2254-4030-b96e-047350f0371d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:42.473476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b9123-2254-4030-b96e-047350f0371d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:42.473747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b9123-2254-4030-b96e-047350f0371d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:42.480437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a6356-6c15-461d-8a74-38ec5bc15d8b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:42.501245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:42.515526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855877e-0d84-49ea-9ecd-15fcee883dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:42.516390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855877e-0d84-49ea-9ecd-15fcee883dd3 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:42.516665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855877e-0d84-49ea-9ecd-15fcee883dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:42.516935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855877e-0d84-49ea-9ecd-15fcee883dd3 None admin] PATH_INFO: `/v3/groups/3bb70362290945a6bcba24363af6df8b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:42.518094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e855877e-0d84-49ea-9ecd-15fcee883dd3 None admin] RBAC: Authorizing `identity:get_group(group_id=3bb70362290945a6bcba24363af6df8b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:42.520160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e855877e-0d84-49ea-9ecd-15fcee883dd3 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:42.521861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 221/450] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:42 2022] GET /identity/v3/groups/3bb70362290945a6bcba24363af6df8b => generated 227 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:42.527585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855877e-0d84-49ea-9ecd-15fcee883dd3 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:42.545188 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b9123-2254-4030-b96e-047350f0371d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:42.546680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b9123-2254-4030-b96e-047350f0371d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:42.604717 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:42.632058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855877e-0d84-49ea-9ecd-15fcee883dd3 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:42.648060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 230/451] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:42 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:39:42.654558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b9123-2254-4030-b96e-047350f0371d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:42.668183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52b7ef-8645-4fb8-9600-5b222590db95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:42.668840 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52b7ef-8645-4fb8-9600-5b222590db95 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:42.669122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52b7ef-8645-4fb8-9600-5b222590db95 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:42.669393 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52b7ef-8645-4fb8-9600-5b222590db95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:42.670218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52b7ef-8645-4fb8-9600-5b222590db95 None admin] RBAC: Adding query filter params (group.id=3bb70362290945a6bcba24363af6df8b, role.id=582f59fc05ce474287c6fdc503f9ac82, scope.project.id=77149f8497f3433c8a1c236067c9eb62) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:42.670527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52b7ef-8645-4fb8-9600-5b222590db95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:42.672004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52b7ef-8645-4fb8-9600-5b222590db95 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:42.678075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 222/452] 10.222.0.121 () {56 vars in 1381 bytes} [Thu May 26 17:39:42 2022] GET /identity/v3/role_assignments?group.id=3bb70362290945a6bcba24363af6df8b&scope.project.id=77149f8497f3433c8a1c236067c9eb62&role.id=582f59fc05ce474287c6fdc503f9ac82 => generated 613 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:42.765190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b9123-2254-4030-b96e-047350f0371d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:42.805164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95a6c0-14b0-4da6-a494-d93363b5de13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:42.806036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95a6c0-14b0-4da6-a494-d93363b5de13 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:42.806318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95a6c0-14b0-4da6-a494-d93363b5de13 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:42.806591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95a6c0-14b0-4da6-a494-d93363b5de13 None admin] PATH_INFO: `/v3/projects/a01a22dec10b499785fdfbdf0a6fcb47` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:42.807862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95a6c0-14b0-4da6-a494-d93363b5de13 None admin] RBAC: Authorizing `identity:get_project(project_id=a01a22dec10b499785fdfbdf0a6fcb47)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:42.809389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95a6c0-14b0-4da6-a494-d93363b5de13 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:42.811187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 231/453] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:39:42 2022] GET /identity/v3/projects/a01a22dec10b499785fdfbdf0a6fcb47 => generated 345 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:42.817899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52b7ef-8645-4fb8-9600-5b222590db95 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:42.874562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:42.923615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52b7ef-8645-4fb8-9600-5b222590db95 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:42.959139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce8e25-0af0-4a16-a9f4-45959d77ecb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:42.960007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8e25-0af0-4a16-a9f4-45959d77ecb9 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:42.960287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8e25-0af0-4a16-a9f4-45959d77ecb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:42.960557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8e25-0af0-4a16-a9f4-45959d77ecb9 None admin] PATH_INFO: `/v3/groups/99468c055e7d412b907043393618a43f` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:42.961728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce8e25-0af0-4a16-a9f4-45959d77ecb9 None admin] RBAC: Authorizing `identity:get_group(group_id=99468c055e7d412b907043393618a43f)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:42.963182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce8e25-0af0-4a16-a9f4-45959d77ecb9 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:42.964798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 223/454] 10.222.0.121 () {56 vars in 1185 bytes} [Thu May 26 17:39:42 2022] GET /identity/v3/groups/99468c055e7d412b907043393618a43f => generated 230 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:42.970606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95a6c0-14b0-4da6-a494-d93363b5de13 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:43.000452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:43.078983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95a6c0-14b0-4da6-a494-d93363b5de13 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:43.111918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e5716-a2ca-4b53-8b72-aa3de458337e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:43.113068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e5716-a2ca-4b53-8b72-aa3de458337e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:43.113404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e5716-a2ca-4b53-8b72-aa3de458337e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:43.113673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e5716-a2ca-4b53-8b72-aa3de458337e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:43.114609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e5716-a2ca-4b53-8b72-aa3de458337e None admin] RBAC: Adding query filter params (group.id=99468c055e7d412b907043393618a43f, role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=a01a22dec10b499785fdfbdf0a6fcb47) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:43.114945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e5716-a2ca-4b53-8b72-aa3de458337e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:43.116381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e5716-a2ca-4b53-8b72-aa3de458337e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:43.122190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 232/455] 10.222.0.121 () {56 vars in 1381 bytes} [Thu May 26 17:39:42 2022] GET /identity/v3/role_assignments?group.id=99468c055e7d412b907043393618a43f&scope.project.id=a01a22dec10b499785fdfbdf0a6fcb47&role.id=4f574961798246a4b12ad2276e67685d => generated 613 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:45.844998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5d428-a150-4313-ab61-7db1525b7293 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:45.845528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5d428-a150-4313-ab61-7db1525b7293 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:45.845528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5d428-a150-4313-ab61-7db1525b7293 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:45.847808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 224/456] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:45.853369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4d523-5802-4911-884f-ad68f3fcc058 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:45.854252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4d523-5802-4911-884f-ad68f3fcc058 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:45.854749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4d523-5802-4911-884f-ad68f3fcc058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:45.887123 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b3d4d523-5802-4911-884f-ad68f3fcc058 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:45.888647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b3d4d523-5802-4911-884f-ad68f3fcc058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:45.979958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 233/457] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:45.995492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb689a0-8048-47a8-b48e-3966ba7ce555 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:45.995825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb689a0-8048-47a8-b48e-3966ba7ce555 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:45.996058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb689a0-8048-47a8-b48e-3966ba7ce555 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:45.997746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 225/458] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:46.003492 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5effd68-e8c6-4055-a19b-b349f9d901c7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:46.003858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5effd68-e8c6-4055-a19b-b349f9d901c7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:46.004139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5effd68-e8c6-4055-a19b-b349f9d901c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:46.013071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb689a0-8048-47a8-b48e-3966ba7ce555 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:46.027147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c5effd68-e8c6-4055-a19b-b349f9d901c7 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:46.028345 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c5effd68-e8c6-4055-a19b-b349f9d901c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:46.055727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:46.134300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 234/459] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:46.160297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5effd68-e8c6-4055-a19b-b349f9d901c7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:46.183837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb689a0-8048-47a8-b48e-3966ba7ce555 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:46.227169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:46.264011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b36548-c89d-4d6e-a82c-0086b8df8759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:46.264831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b36548-c89d-4d6e-a82c-0086b8df8759 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:46.265118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b36548-c89d-4d6e-a82c-0086b8df8759 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:46.265397 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b36548-c89d-4d6e-a82c-0086b8df8759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:46.287846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b36548-c89d-4d6e-a82c-0086b8df8759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:46.289800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b36548-c89d-4d6e-a82c-0086b8df8759 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:46.298603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5effd68-e8c6-4055-a19b-b349f9d901c7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-72b36548-c89d-4d6e-a82c-0086b8df8759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:46.329410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:46.330341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:46.331257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:46.331257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:46.331257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:46.331257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:46.333626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 226/460] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:46.338390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604c9ff-b74b-4368-a13c-c6b2d625f1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:46.339286 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604c9ff-b74b-4368-a13c-c6b2d625f1b2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:46.339571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604c9ff-b74b-4368-a13c-c6b2d625f1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:46.339843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604c9ff-b74b-4368-a13c-c6b2d625f1b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:46.340537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b36548-c89d-4d6e-a82c-0086b8df8759 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:46.352775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604c9ff-b74b-4368-a13c-c6b2d625f1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:46.354434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604c9ff-b74b-4368-a13c-c6b2d625f1b2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7604c9ff-b74b-4368-a13c-c6b2d625f1b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:46.374334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:46.375275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:46.376097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:46.376097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:46.376097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:46.376097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:46.376097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 235/461] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:46.382507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae58da-a7a6-4b5a-a616-15561c322bf5 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:46.382790 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae58da-a7a6-4b5a-a616-15561c322bf5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:46.383100 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae58da-a7a6-4b5a-a616-15561c322bf5 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:46.384738 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 236/462] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:39:46.385460 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae58da-a7a6-4b5a-a616-15561c322bf5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:46.397960 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:46.475537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b36548-c89d-4d6e-a82c-0086b8df8759 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:46.500220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae58da-a7a6-4b5a-a616-15561c322bf5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:46.511591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d04b0f-56d4-414f-91a5-8b5453ee1dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:46.512539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d04b0f-56d4-414f-91a5-8b5453ee1dea None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:46.513020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d04b0f-56d4-414f-91a5-8b5453ee1dea None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:46.513541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d04b0f-56d4-414f-91a5-8b5453ee1dea None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:46.514500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d04b0f-56d4-414f-91a5-8b5453ee1dea None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:46.515073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d04b0f-56d4-414f-91a5-8b5453ee1dea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:46.516699 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d04b0f-56d4-414f-91a5-8b5453ee1dea None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:46.536457 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 227/463] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:46.538127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a183157-b757-46d7-8a93-f916d6506c40 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:46.538571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f481027-c495-4f8b-9202-5232a51abf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:46.538889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a183157-b757-46d7-8a93-f916d6506c40 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:46.539284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f481027-c495-4f8b-9202-5232a51abf9c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:46.539568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f481027-c495-4f8b-9202-5232a51abf9c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:46.539845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f481027-c495-4f8b-9202-5232a51abf9c None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:46.540116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a183157-b757-46d7-8a93-f916d6506c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:46.540674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f481027-c495-4f8b-9202-5232a51abf9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:46.540982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f481027-c495-4f8b-9202-5232a51abf9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:46.542636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f481027-c495-4f8b-9202-5232a51abf9c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:46.556164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 237/464] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:46.557802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1022b41-cdb1-4f19-a138-fc69f6d5cc8a None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:46.558372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1022b41-cdb1-4f19-a138-fc69f6d5cc8a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:46.558646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1022b41-cdb1-4f19-a138-fc69f6d5cc8a None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:46.560044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 238/465] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:46.561638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3a183157-b757-46d7-8a93-f916d6506c40 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:46.563182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3a183157-b757-46d7-8a93-f916d6506c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:46.563695 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1022b41-cdb1-4f19-a138-fc69f6d5cc8a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:46.592147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:46.634292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:46.651009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 228/466] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:46.651957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a183157-b757-46d7-8a93-f916d6506c40 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:46.681779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1022b41-cdb1-4f19-a138-fc69f6d5cc8a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:46.719714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76228d3-333e-4019-82ba-b32f6a4d950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:46.720390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76228d3-333e-4019-82ba-b32f6a4d950e None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:46.720697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76228d3-333e-4019-82ba-b32f6a4d950e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:46.720967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76228d3-333e-4019-82ba-b32f6a4d950e None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:46.721667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76228d3-333e-4019-82ba-b32f6a4d950e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:46.723176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76228d3-333e-4019-82ba-b32f6a4d950e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:46.760887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a183157-b757-46d7-8a93-f916d6506c40 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:46.797096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7aac82-a7fc-4009-92db-a2f142fa9a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:46.797858 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7aac82-a7fc-4009-92db-a2f142fa9a54 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:46.798144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7aac82-a7fc-4009-92db-a2f142fa9a54 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:46.798422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7aac82-a7fc-4009-92db-a2f142fa9a54 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:46.799155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7aac82-a7fc-4009-92db-a2f142fa9a54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:46.801031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7aac82-a7fc-4009-92db-a2f142fa9a54 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:46.835331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 239/467] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:39:46 2022] POST /identity/v3/users => generated 247 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:39:46.836737 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74674ab-66f8-4c3b-9dde-f28b35e34b01 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:46.837027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74674ab-66f8-4c3b-9dde-f28b35e34b01 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:46.837350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74674ab-66f8-4c3b-9dde-f28b35e34b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:46.856572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d74674ab-66f8-4c3b-9dde-f28b35e34b01 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:46.857806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d74674ab-66f8-4c3b-9dde-f28b35e34b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:46.922748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 229/468] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:39:46 2022] POST /identity/v3/users => generated 249 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:39:46.924590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7aac82-a7fc-4009-92db-a2f142fa9a54 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:46.959578 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 240/469] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:46.967330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:46.984511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74674ab-66f8-4c3b-9dde-f28b35e34b01 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:47.099388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7aac82-a7fc-4009-92db-a2f142fa9a54 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:47.100455 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:47.153929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a8d3e-8355-4b46-b673-98262dbf4019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:47.154740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8d3e-8355-4b46-b673-98262dbf4019 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:47.156475 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8d3e-8355-4b46-b673-98262dbf4019 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:47.156475 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8d3e-8355-4b46-b673-98262dbf4019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:47.169073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a8d3e-8355-4b46-b673-98262dbf4019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:47.170890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a8d3e-8355-4b46-b673-98262dbf4019 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:47.174969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74674ab-66f8-4c3b-9dde-f28b35e34b01 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-5b7a8d3e-8355-4b46-b673-98262dbf4019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:47.199929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:47.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:47.201694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:47.201694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:47.201694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:47.201694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:47.201694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 230/470] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:47.207935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a8d3e-8355-4b46-b673-98262dbf4019 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:47.232969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd65f6-6eb5-4ea7-8caf-f1149051f02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:47.233779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd65f6-6eb5-4ea7-8caf-f1149051f02e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:47.234074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd65f6-6eb5-4ea7-8caf-f1149051f02e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:47.234328 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd65f6-6eb5-4ea7-8caf-f1149051f02e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:47.248587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd65f6-6eb5-4ea7-8caf-f1149051f02e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:47.250053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd65f6-6eb5-4ea7-8caf-f1149051f02e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f4dd65f6-6eb5-4ea7-8caf-f1149051f02e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:47.276268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:47.277331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:47.278588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:47.278588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:47.278588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:47.278588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:47.278588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 241/471] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:47.283615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd65f6-6eb5-4ea7-8caf-f1149051f02e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:47.311203 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:47.342857 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a8d3e-8355-4b46-b673-98262dbf4019 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:47.385300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ac3ef-0244-471b-87ca-513ef9398619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:47.386115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ac3ef-0244-471b-87ca-513ef9398619 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:47.386400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ac3ef-0244-471b-87ca-513ef9398619 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:47.386678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ac3ef-0244-471b-87ca-513ef9398619 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:47.387453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ac3ef-0244-471b-87ca-513ef9398619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:47.387765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ac3ef-0244-471b-87ca-513ef9398619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:47.389478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ac3ef-0244-471b-87ca-513ef9398619 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:47.403586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 231/472] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:47.410171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ac3ef-0244-471b-87ca-513ef9398619 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:47.515569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:47.518376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd65f6-6eb5-4ea7-8caf-f1149051f02e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:47.530845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ac3ef-0244-471b-87ca-513ef9398619 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:47.559823 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:47.564688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f33ad5-540d-453b-8a4f-67bc32925691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:47.565538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f33ad5-540d-453b-8a4f-67bc32925691 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:47.566071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f33ad5-540d-453b-8a4f-67bc32925691 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:47.566427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f33ad5-540d-453b-8a4f-67bc32925691 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:47.567298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f33ad5-540d-453b-8a4f-67bc32925691 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:47.570853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f33ad5-540d-453b-8a4f-67bc32925691 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:47.587691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4cea77-2261-4f04-86c7-efd08d0ada1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:47.588464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4cea77-2261-4f04-86c7-efd08d0ada1d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:47.588744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4cea77-2261-4f04-86c7-efd08d0ada1d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:47.589016 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4cea77-2261-4f04-86c7-efd08d0ada1d None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:47.589747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4cea77-2261-4f04-86c7-efd08d0ada1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:47.590081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4cea77-2261-4f04-86c7-efd08d0ada1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:47.591484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4cea77-2261-4f04-86c7-efd08d0ada1d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:47.603687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 242/473] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:47.604995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89064ff-8e62-4de2-bb25-dff18b8878a7 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:47.605287 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89064ff-8e62-4de2-bb25-dff18b8878a7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:47.605568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89064ff-8e62-4de2-bb25-dff18b8878a7 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:47.606798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 243/474] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:47.611958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89064ff-8e62-4de2-bb25-dff18b8878a7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:47.723182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 232/475] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:39:47 2022] POST /identity/v3/users => generated 249 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:39:47.724952 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc1d36-4ef6-47c2-bee6-c88466813276 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:47.725244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc1d36-4ef6-47c2-bee6-c88466813276 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:47.725526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc1d36-4ef6-47c2-bee6-c88466813276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:47.735500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89064ff-8e62-4de2-bb25-dff18b8878a7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:47.776150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c0cc1d36-4ef6-47c2-bee6-c88466813276 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:47.777238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c0cc1d36-4ef6-47c2-bee6-c88466813276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:47.785986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88354632-676c-42b1-8301-f6d6367ad7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:47.787067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88354632-676c-42b1-8301-f6d6367ad7ee None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:47.787642 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88354632-676c-42b1-8301-f6d6367ad7ee None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:47.788116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88354632-676c-42b1-8301-f6d6367ad7ee None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:47.789067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88354632-676c-42b1-8301-f6d6367ad7ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:47.790951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88354632-676c-42b1-8301-f6d6367ad7ee None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:47.875494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 233/476] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:47 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:47.900320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc1d36-4ef6-47c2-bee6-c88466813276 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:47.918800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 244/477] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:39:47 2022] POST /identity/v3/users => generated 250 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:39:48.058447 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc1d36-4ef6-47c2-bee6-c88466813276 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:48.084383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:48.120744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154dd31-b46e-41a4-9977-9303f1d1fcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:48.121847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154dd31-b46e-41a4-9977-9303f1d1fcaf None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:48.127390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154dd31-b46e-41a4-9977-9303f1d1fcaf None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:48.128316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154dd31-b46e-41a4-9977-9303f1d1fcaf None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:48.129275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154dd31-b46e-41a4-9977-9303f1d1fcaf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:48.130865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154dd31-b46e-41a4-9977-9303f1d1fcaf None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:48.196778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.sql.core [None req-3154dd31-b46e-41a4-9977-9303f1d1fcaf None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 17:39:48.196778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 17:39:48.196778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [parameters: {'id': '8529c317bafb41baac981497ba8d7158', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 17:39:48.196778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92082) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-3154dd31-b46e-41a4-9977-9303f1d1fcaf None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application result = self._query(query) May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application conn.query(q) May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application result.read() May 26 17:39:48.215345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return ref.to_dict() May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 17:39:48.216503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application next(self.gen) May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application yield resource May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application next(self.gen) May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self.session.rollback() May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self.force_reraise() May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise self.value May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application session.commit() May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 17:39:48.217635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._prepare_impl() May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self.session.flush() May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._flush(objects) May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application compat.raise_( May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application flush_context.execute() May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rec.execute(self) May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 17:39:48.218684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 17:39:48.219891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application result = self._query(query) May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application conn.query(q) May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application result.read() May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 17:39:48.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [parameters: {'id': '8529c317bafb41baac981497ba8d7158', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:48.222563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 17:39:48.223769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:48.224656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 234/478] 10.222.0.121 () {60 vars in 1136 bytes} [Thu May 26 17:39:47 2022] POST /identity/v3/roles => generated 144 bytes in 325 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:48.231303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88354632-676c-42b1-8301-f6d6367ad7ee None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:48.257008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:48.358482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88354632-676c-42b1-8301-f6d6367ad7ee None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:48.396604 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a2f06-ee57-4b54-a7d4-8f99a4f100d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:48.397797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a2f06-ee57-4b54-a7d4-8f99a4f100d9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:48.398222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a2f06-ee57-4b54-a7d4-8f99a4f100d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:48.398690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a2f06-ee57-4b54-a7d4-8f99a4f100d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:48.410378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a2f06-ee57-4b54-a7d4-8f99a4f100d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-152a2f06-ee57-4b54-a7d4-8f99a4f100d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:48.419221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:48.420324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:48.421278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:48.421278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:48.421278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:48.421278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:48.421278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:48.421278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:48.421278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:48.421278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:48.421278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:48.421278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:48.421278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:39:48.421278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:48.423718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 245/479] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:39:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:48.432205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154dd31-b46e-41a4-9977-9303f1d1fcaf None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:48.546060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154dd31-b46e-41a4-9977-9303f1d1fcaf None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:48.584478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bf641-6b7d-41e7-84ab-a8a5a06b143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:48.585651 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bf641-6b7d-41e7-84ab-a8a5a06b143f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:48.586113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bf641-6b7d-41e7-84ab-a8a5a06b143f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:48.586593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bf641-6b7d-41e7-84ab-a8a5a06b143f None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:48.587615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bf641-6b7d-41e7-84ab-a8a5a06b143f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:48.588098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bf641-6b7d-41e7-84ab-a8a5a06b143f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:48.589693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bf641-6b7d-41e7-84ab-a8a5a06b143f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:48.599881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 235/480] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:48 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:49.621140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591f880-264b-4e75-b7b4-7b5757944a36 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:49.621660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591f880-264b-4e75-b7b4-7b5757944a36 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:49.621736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591f880-264b-4e75-b7b4-7b5757944a36 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:49.623387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 246/481] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:49.632283 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cad0f-2395-4935-9750-f55de379d32f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:49.632758 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cad0f-2395-4935-9750-f55de379d32f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:49.632832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cad0f-2395-4935-9750-f55de379d32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:49.655335 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-ea3cad0f-2395-4935-9750-f55de379d32f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:49.656525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ea3cad0f-2395-4935-9750-f55de379d32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:49.687327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bb2a0-70cd-40c4-8a5f-e03a839fde21 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:49.687634 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bb2a0-70cd-40c4-8a5f-e03a839fde21 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:49.687882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bb2a0-70cd-40c4-8a5f-e03a839fde21 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:49.689499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 247/482] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:49.695179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b042bed-c8b6-486a-8e22-90c1205e6ab7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:49.695463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b042bed-c8b6-486a-8e22-90c1205e6ab7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:49.695738 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b042bed-c8b6-486a-8e22-90c1205e6ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:49.723556 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:49.730895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1b042bed-c8b6-486a-8e22-90c1205e6ab7 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:49.732105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1b042bed-c8b6-486a-8e22-90c1205e6ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:49.778175 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 236/483] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:49.802650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cad0f-2395-4935-9750-f55de379d32f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:49.860346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 248/484] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:49.901203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b042bed-c8b6-486a-8e22-90c1205e6ab7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:49.910207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:49.969425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cad0f-2395-4935-9750-f55de379d32f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:50.042498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe6787-9d75-400a-bb61-d30f566aef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:50.043339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe6787-9d75-400a-bb61-d30f566aef62 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:50.043620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe6787-9d75-400a-bb61-d30f566aef62 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:50.043891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe6787-9d75-400a-bb61-d30f566aef62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:50.056237 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe6787-9d75-400a-bb61-d30f566aef62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-aebe6787-9d75-400a-bb61-d30f566aef62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:50.064031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:50.065040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:50.066417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 237/485] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:39:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:50.072060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe6787-9d75-400a-bb61-d30f566aef62 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:50.091919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b042bed-c8b6-486a-8e22-90c1205e6ab7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:50.139191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:50.164749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a801b6-ace8-423d-b278-8fd23c2ed121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:50.165928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a801b6-ace8-423d-b278-8fd23c2ed121 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:50.166686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a801b6-ace8-423d-b278-8fd23c2ed121 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:50.167179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a801b6-ace8-423d-b278-8fd23c2ed121 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:50.182302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a801b6-ace8-423d-b278-8fd23c2ed121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-53a801b6-ace8-423d-b278-8fd23c2ed121 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:50.195856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:50.196800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:50.197588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:50.197588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:50.197588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:50.197588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:50.197588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:50.197588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:50.197588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:50.197588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:50.197588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:50.197588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:50.197588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:50.197588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:50.200837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 249/486] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:39:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:50.209409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a801b6-ace8-423d-b278-8fd23c2ed121 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:50.252500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe6787-9d75-400a-bb61-d30f566aef62 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:50.292442 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450af154-cbbc-4261-bd42-379ba4494878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:50.293130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450af154-cbbc-4261-bd42-379ba4494878 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:50.293417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450af154-cbbc-4261-bd42-379ba4494878 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:50.293689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450af154-cbbc-4261-bd42-379ba4494878 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:50.294456 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450af154-cbbc-4261-bd42-379ba4494878 None admin] RBAC: Adding query filter params (name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:50.294759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450af154-cbbc-4261-bd42-379ba4494878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:50.296200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450af154-cbbc-4261-bd42-379ba4494878 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:50.310726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 238/487] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:39:50 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:50.312016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450af154-cbbc-4261-bd42-379ba4494878 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:50.363229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:50.415387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:50.422579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450af154-cbbc-4261-bd42-379ba4494878 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:50.455270 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a801b6-ace8-423d-b278-8fd23c2ed121 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:50.469150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b9d91-1c1b-4e69-92eb-ff82f810b193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:50.470356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b9d91-1c1b-4e69-92eb-ff82f810b193 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:50.470639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b9d91-1c1b-4e69-92eb-ff82f810b193 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:50.470902 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b9d91-1c1b-4e69-92eb-ff82f810b193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:50.484939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b9d91-1c1b-4e69-92eb-ff82f810b193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:50.486388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b9d91-1c1b-4e69-92eb-ff82f810b193 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:50.504859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b310df9-970b-4404-9a11-8cf9e0cd45b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:50.505525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b310df9-970b-4404-9a11-8cf9e0cd45b1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:50.505800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b310df9-970b-4404-9a11-8cf9e0cd45b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:50.506067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b310df9-970b-4404-9a11-8cf9e0cd45b1 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:50.506835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b310df9-970b-4404-9a11-8cf9e0cd45b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:50.507156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b310df9-970b-4404-9a11-8cf9e0cd45b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:50.508537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b310df9-970b-4404-9a11-8cf9e0cd45b1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-1f6b9d91-1c1b-4e69-92eb-ff82f810b193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:50.511756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:50.512607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:50.513454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:50.513454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:50.513454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:50.513454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:50.513454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 239/488] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:50.514926 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cbd83-e12f-4bbe-9661-d09b6cb329e4 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:50.515476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cbd83-e12f-4bbe-9661-d09b6cb329e4 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:50.515817 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cbd83-e12f-4bbe-9661-d09b6cb329e4 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:50.522125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 240/489] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:50 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:50.523306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cbd83-e12f-4bbe-9661-d09b6cb329e4 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:50.523798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 250/490] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:39:50 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:50.525103 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a08a5f-3103-4b68-aa85-8076b431e45f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:50.525386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a08a5f-3103-4b68-aa85-8076b431e45f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:50.525657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a08a5f-3103-4b68-aa85-8076b431e45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:50.552760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-96a08a5f-3103-4b68-aa85-8076b431e45f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:50.554105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-96a08a5f-3103-4b68-aa85-8076b431e45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:50.641209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cbd83-e12f-4bbe-9661-d09b6cb329e4 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:50.641876 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 251/491] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:50.642310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a08a5f-3103-4b68-aa85-8076b431e45f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:50.698660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44870696-2690-4453-9da1-60217996e010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:50.699412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44870696-2690-4453-9da1-60217996e010 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:50.699689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44870696-2690-4453-9da1-60217996e010 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:50.699959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44870696-2690-4453-9da1-60217996e010 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:50.700682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44870696-2690-4453-9da1-60217996e010 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:50.700983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44870696-2690-4453-9da1-60217996e010 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:50.702339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44870696-2690-4453-9da1-60217996e010 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:50.709782 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:50.716005 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 241/492] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:50.716877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44870696-2690-4453-9da1-60217996e010 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:50.732474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:50.773364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a08a5f-3103-4b68-aa85-8076b431e45f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:50.817852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ca1f3-f192-491e-b9ad-3e638a76ef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:50.818758 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ca1f3-f192-491e-b9ad-3e638a76ef3a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:50.819063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ca1f3-f192-491e-b9ad-3e638a76ef3a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:50.819331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ca1f3-f192-491e-b9ad-3e638a76ef3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:50.830783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ca1f3-f192-491e-b9ad-3e638a76ef3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:50.832910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ca1f3-f192-491e-b9ad-3e638a76ef3a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:50.844548 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44870696-2690-4453-9da1-60217996e010 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-305ca1f3-f192-491e-b9ad-3e638a76ef3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:50.853113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:50.855193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:50.857235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:50.857235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:50.857235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:50.857235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:50.862856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 252/493] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:50.862856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ca1f3-f192-491e-b9ad-3e638a76ef3a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:50.884642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e947d2-c756-4054-9d8b-45cf185f02b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:50.885437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e947d2-c756-4054-9d8b-45cf185f02b4 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:50.885714 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e947d2-c756-4054-9d8b-45cf185f02b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:50.885984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e947d2-c756-4054-9d8b-45cf185f02b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:50.897879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e947d2-c756-4054-9d8b-45cf185f02b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-45e947d2-c756-4054-9d8b-45cf185f02b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:50.906042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:50.908167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 242/494] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:39:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:50.911145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e947d2-c756-4054-9d8b-45cf185f02b4 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:50.969683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ca1f3-f192-491e-b9ad-3e638a76ef3a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:50.982649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:50.983891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:51.008319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee73c84-058c-4e3e-b53a-876e65fb0fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:51.009115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee73c84-058c-4e3e-b53a-876e65fb0fa2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.009452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee73c84-058c-4e3e-b53a-876e65fb0fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.009718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee73c84-058c-4e3e-b53a-876e65fb0fa2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.017581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee73c84-058c-4e3e-b53a-876e65fb0fa2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:51.019784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee73c84-058c-4e3e-b53a-876e65fb0fa2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:51.021547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e947d2-c756-4054-9d8b-45cf185f02b4 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-dee73c84-058c-4e3e-b53a-876e65fb0fa2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:39:51.036372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:51.037331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:51.043192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:51.043192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:51.043192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:39:51.043192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:39:51.043192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:39:51.043192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:39:51.043192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:39:51.043192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:39:51.043192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 17:39:51.043192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:51.043192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 253/495] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:51.043192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee73c84-058c-4e3e-b53a-876e65fb0fa2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:51.065002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15f96f-340c-446f-94ca-408a83ed0dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:51.066250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15f96f-340c-446f-94ca-408a83ed0dcf None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.066521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15f96f-340c-446f-94ca-408a83ed0dcf None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.066799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15f96f-340c-446f-94ca-408a83ed0dcf None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.067556 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15f96f-340c-446f-94ca-408a83ed0dcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:51.067860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15f96f-340c-446f-94ca-408a83ed0dcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:51.069242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15f96f-340c-446f-94ca-408a83ed0dcf None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:51.083211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 243/496] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:51.084171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d86cb-16c4-48f4-87d7-96ef008077ae None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.084466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d86cb-16c4-48f4-87d7-96ef008077ae None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.084744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d86cb-16c4-48f4-87d7-96ef008077ae None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.086206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 244/497] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:51.086847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d86cb-16c4-48f4-87d7-96ef008077ae None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:51.173401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee73c84-058c-4e3e-b53a-876e65fb0fa2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:51.205423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d86cb-16c4-48f4-87d7-96ef008077ae None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:51.208463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6c76e-afd3-4a4a-8f9a-0050f46645c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:51.209191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6c76e-afd3-4a4a-8f9a-0050f46645c6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.209464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6c76e-afd3-4a4a-8f9a-0050f46645c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.209733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6c76e-afd3-4a4a-8f9a-0050f46645c6 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.210477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6c76e-afd3-4a4a-8f9a-0050f46645c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:51.210777 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6c76e-afd3-4a4a-8f9a-0050f46645c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:51.212193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6c76e-afd3-4a4a-8f9a-0050f46645c6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:51.220455 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 254/498] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:39:51 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:51.221192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6c76e-afd3-4a4a-8f9a-0050f46645c6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:51.239834 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:51.249149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfee6ce-0d3a-4348-a224-25dc1c9ae594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:51.250180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfee6ce-0d3a-4348-a224-25dc1c9ae594 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.250760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfee6ce-0d3a-4348-a224-25dc1c9ae594 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.251077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfee6ce-0d3a-4348-a224-25dc1c9ae594 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.251918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfee6ce-0d3a-4348-a224-25dc1c9ae594 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:51.252315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfee6ce-0d3a-4348-a224-25dc1c9ae594 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:51.254846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfee6ce-0d3a-4348-a224-25dc1c9ae594 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:51.257856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:51.301536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 245/499] 10.222.0.121 () {56 vars in 1142 bytes} [Thu May 26 17:39:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:51.301536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b45ea-be9d-4020-aa6c-7e3ed2d8758c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.301536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b45ea-be9d-4020-aa6c-7e3ed2d8758c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.301536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b45ea-be9d-4020-aa6c-7e3ed2d8758c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.318103 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-7b4b45ea-be9d-4020-aa6c-7e3ed2d8758c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:51.319343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7b4b45ea-be9d-4020-aa6c-7e3ed2d8758c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:51.338310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6c76e-afd3-4a4a-8f9a-0050f46645c6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:51.374790 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a82303-7f43-463b-b830-342e6a8e7ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:51.375623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a82303-7f43-463b-b830-342e6a8e7ef8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.375902 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a82303-7f43-463b-b830-342e6a8e7ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.376173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a82303-7f43-463b-b830-342e6a8e7ef8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.384131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a82303-7f43-463b-b830-342e6a8e7ef8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:51.385562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a82303-7f43-463b-b830-342e6a8e7ef8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:51.398853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 246/500] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:51.398853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b45ea-be9d-4020-aa6c-7e3ed2d8758c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-b9a82303-7f43-463b-b830-342e6a8e7ef8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:39:51.408456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:51.412115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:51.413629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:51.413629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:51.413629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:39:51.413629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:39:51.413629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:39:51.413629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:39:51.413629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:39:51.413629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:39:51.413629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:39:51.413629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:51.413629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 255/501] 10.222.0.121 () {56 vars in 1098 bytes} [Thu May 26 17:39:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:51.413629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a82303-7f43-463b-b830-342e6a8e7ef8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:51.514413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b45ea-be9d-4020-aa6c-7e3ed2d8758c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:51.523174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:51.531238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:51.532237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a82303-7f43-463b-b830-342e6a8e7ef8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:51.552820 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f4637-c074-4e64-9c29-a3c31cd2332a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:51.553624 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f4637-c074-4e64-9c29-a3c31cd2332a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.553901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f4637-c074-4e64-9c29-a3c31cd2332a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.554337 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f4637-c074-4e64-9c29-a3c31cd2332a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.567847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f4637-c074-4e64-9c29-a3c31cd2332a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:51.569413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f4637-c074-4e64-9c29-a3c31cd2332a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:51.570446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a91e3-e0a1-4bd5-9a0a-d137896af8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:51.571207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a91e3-e0a1-4bd5-9a0a-d137896af8b3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.571413 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a91e3-e0a1-4bd5-9a0a-d137896af8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.572313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a91e3-e0a1-4bd5-9a0a-d137896af8b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-4e9f4637-c074-4e64-9c29-a3c31cd2332a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:51.588064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:51.588977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:51.589823 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:51.589823 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:51.589823 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:51.589823 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:51.589823 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 247/502] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:51.590080 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f4637-c074-4e64-9c29-a3c31cd2332a None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:51.653360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a91e3-e0a1-4bd5-9a0a-d137896af8b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:51.654914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a91e3-e0a1-4bd5-9a0a-d137896af8b3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:51.699489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f4637-c074-4e64-9c29-a3c31cd2332a None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:51.733871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08816cfc-1a1e-4a71-b3a1-aa4f1a72ddb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:51.742225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08816cfc-1a1e-4a71-b3a1-aa4f1a72ddb2 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.742225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08816cfc-1a1e-4a71-b3a1-aa4f1a72ddb2 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.742225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08816cfc-1a1e-4a71-b3a1-aa4f1a72ddb2 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.742225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08816cfc-1a1e-4a71-b3a1-aa4f1a72ddb2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:51.742225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08816cfc-1a1e-4a71-b3a1-aa4f1a72ddb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:51.742225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08816cfc-1a1e-4a71-b3a1-aa4f1a72ddb2 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:51.747161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:51.756659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 248/503] 10.222.0.121 () {56 vars in 1138 bytes} [Thu May 26 17:39:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:51.757509 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08816cfc-1a1e-4a71-b3a1-aa4f1a72ddb2 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:51.761721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 256/504] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:51 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:51.762439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a91e3-e0a1-4bd5-9a0a-d137896af8b3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:51.783104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:51.859687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08816cfc-1a1e-4a71-b3a1-aa4f1a72ddb2 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:51.865109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a91e3-e0a1-4bd5-9a0a-d137896af8b3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:51.895930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74720c2a-7a41-440f-80e0-fbd86ec59364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:51.896729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74720c2a-7a41-440f-80e0-fbd86ec59364 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.897008 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74720c2a-7a41-440f-80e0-fbd86ec59364 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.897282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74720c2a-7a41-440f-80e0-fbd86ec59364 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.901069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae45e9-2a7d-4351-a148-47ce46db5a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:51.901729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae45e9-2a7d-4351-a148-47ce46db5a9d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.902011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae45e9-2a7d-4351-a148-47ce46db5a9d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.902281 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae45e9-2a7d-4351-a148-47ce46db5a9d None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.903028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae45e9-2a7d-4351-a148-47ce46db5a9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:51.903332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae45e9-2a7d-4351-a148-47ce46db5a9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:51.904731 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae45e9-2a7d-4351-a148-47ce46db5a9d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:51.907424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74720c2a-7a41-440f-80e0-fbd86ec59364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:51.918366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 257/505] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:51.919747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8725f-b9a5-4148-8eb9-b4ec44cd36f0 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:51.920033 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8725f-b9a5-4148-8eb9-b4ec44cd36f0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:51.920304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8725f-b9a5-4148-8eb9-b4ec44cd36f0 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:51.922861 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 258/506] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:51.922861 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8725f-b9a5-4148-8eb9-b4ec44cd36f0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-74720c2a-7a41-440f-80e0-fbd86ec59364 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:51.923077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:51.923941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:51.925374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 249/507] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:39:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:51.926161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74720c2a-7a41-440f-80e0-fbd86ec59364 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:51.981708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:52.015148 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:52.032109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74720c2a-7a41-440f-80e0-fbd86ec59364 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:52.033947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8725f-b9a5-4148-8eb9-b4ec44cd36f0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:52.068728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719f8ff-0663-48b2-b0e8-10bcc8a1e1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:52.068958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ddb70-f6fa-4f01-b150-5898054fa7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:52.069502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ddb70-f6fa-4f01-b150-5898054fa7ea None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:52.069599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719f8ff-0663-48b2-b0e8-10bcc8a1e1b1 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:52.069778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ddb70-f6fa-4f01-b150-5898054fa7ea None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:52.069865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719f8ff-0663-48b2-b0e8-10bcc8a1e1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:52.070045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ddb70-f6fa-4f01-b150-5898054fa7ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:52.070140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719f8ff-0663-48b2-b0e8-10bcc8a1e1b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:52.080421 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719f8ff-0663-48b2-b0e8-10bcc8a1e1b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:52.082585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719f8ff-0663-48b2-b0e8-10bcc8a1e1b1 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-9719f8ff-0663-48b2-b0e8-10bcc8a1e1b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:52.104470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:52.105377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:52.106198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:52.106198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:52.106198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:52.106198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:52.106198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 250/508] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:52.106965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719f8ff-0663-48b2-b0e8-10bcc8a1e1b1 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:52.145078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ddb70-f6fa-4f01-b150-5898054fa7ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:52.146549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ddb70-f6fa-4f01-b150-5898054fa7ea None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:52.199233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:52.210619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719f8ff-0663-48b2-b0e8-10bcc8a1e1b1 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:52.245658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b847829-8f37-4dd8-a12f-2641cd02c80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:52.246461 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b847829-8f37-4dd8-a12f-2641cd02c80b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:52.246744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b847829-8f37-4dd8-a12f-2641cd02c80b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:52.247050 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b847829-8f37-4dd8-a12f-2641cd02c80b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:52.247807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 259/509] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:51 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:52.249118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d72aa-a1d4-46b0-9e94-c9dbcd68cb0c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:52.249406 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d72aa-a1d4-46b0-9e94-c9dbcd68cb0c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:52.249686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d72aa-a1d4-46b0-9e94-c9dbcd68cb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:52.253262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:52.257971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b847829-8f37-4dd8-a12f-2641cd02c80b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:52.259455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b847829-8f37-4dd8-a12f-2641cd02c80b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:52.267099 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ad8d72aa-a1d4-46b0-9e94-c9dbcd68cb0c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:52.269064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ad8d72aa-a1d4-46b0-9e94-c9dbcd68cb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-6b847829-8f37-4dd8-a12f-2641cd02c80b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:39:52.275602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:52.276555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:52.277443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:52.277443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:52.277443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:39:52.277443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:39:52.277443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:39:52.277443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:39:52.277443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:39:52.277443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:39:52.277443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 17:39:52.277443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:52.277443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 251/510] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:39:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:52.278206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b847829-8f37-4dd8-a12f-2641cd02c80b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:52.341838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 260/511] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:52.342645 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d72aa-a1d4-46b0-9e94-c9dbcd68cb0c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:52.397196 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b847829-8f37-4dd8-a12f-2641cd02c80b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:52.425130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:52.433100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16f1c1-ead7-4e2e-923e-75c5e32c0ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:52.433767 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16f1c1-ead7-4e2e-923e-75c5e32c0ea8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:52.434051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16f1c1-ead7-4e2e-923e-75c5e32c0ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:52.437789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16f1c1-ead7-4e2e-923e-75c5e32c0ea8 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:52.437789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16f1c1-ead7-4e2e-923e-75c5e32c0ea8 None admin] RBAC: Adding query filter params (name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:52.437789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16f1c1-ead7-4e2e-923e-75c5e32c0ea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:52.437789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16f1c1-ead7-4e2e-923e-75c5e32c0ea8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:52.446045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 252/512] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:39:52 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:52.446598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d72aa-a1d4-46b0-9e94-c9dbcd68cb0c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:52.446885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16f1c1-ead7-4e2e-923e-75c5e32c0ea8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:52.485012 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5cef7-0511-458e-9a6f-d5108b43e0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:52.485673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5cef7-0511-458e-9a6f-d5108b43e0ad None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:52.485950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5cef7-0511-458e-9a6f-d5108b43e0ad None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:52.486221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5cef7-0511-458e-9a6f-d5108b43e0ad None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:52.486984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5cef7-0511-458e-9a6f-d5108b43e0ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:52.487432 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5cef7-0511-458e-9a6f-d5108b43e0ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:52.488825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5cef7-0511-458e-9a6f-d5108b43e0ad None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:52.500108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 261/513] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:52.501050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5cef7-0511-458e-9a6f-d5108b43e0ad None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:52.507027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:52.550731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16f1c1-ead7-4e2e-923e-75c5e32c0ea8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:52.585916 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5a9a2-b0d7-408c-88d8-b54c61536252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:52.586768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5a9a2-b0d7-408c-88d8-b54c61536252 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:52.587086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5a9a2-b0d7-408c-88d8-b54c61536252 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:52.587359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5a9a2-b0d7-408c-88d8-b54c61536252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:52.598309 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5a9a2-b0d7-408c-88d8-b54c61536252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:52.599724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5a9a2-b0d7-408c-88d8-b54c61536252 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:52.601387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5cef7-0511-458e-9a6f-d5108b43e0ad None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-6fb5a9a2-b0d7-408c-88d8-b54c61536252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:52.616908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:52.617877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:52.618703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:52.618703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:52.618703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:52.618703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:52.618703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 253/514] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:52.619001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5a9a2-b0d7-408c-88d8-b54c61536252 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:52.641836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a7e3c-cc7d-418c-b3ab-e39511f3d989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:52.642507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a7e3c-cc7d-418c-b3ab-e39511f3d989 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:52.642780 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a7e3c-cc7d-418c-b3ab-e39511f3d989 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:52.643081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a7e3c-cc7d-418c-b3ab-e39511f3d989 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:52.644475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a7e3c-cc7d-418c-b3ab-e39511f3d989 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:52.644773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a7e3c-cc7d-418c-b3ab-e39511f3d989 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:52.646205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a7e3c-cc7d-418c-b3ab-e39511f3d989 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:52.680378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:52.684316 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 262/515] 10.222.0.121 () {56 vars in 1144 bytes} [Thu May 26 17:39:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:52.685251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a7e3c-cc7d-418c-b3ab-e39511f3d989 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:52.724612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5a9a2-b0d7-408c-88d8-b54c61536252 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:52.735173 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:52.760838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb7974-a8c6-4578-9120-c9d44062bd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:52.761716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb7974-a8c6-4578-9120-c9d44062bd68 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:52.761996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb7974-a8c6-4578-9120-c9d44062bd68 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:52.762686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb7974-a8c6-4578-9120-c9d44062bd68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:52.774458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb7974-a8c6-4578-9120-c9d44062bd68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:52.775961 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb7974-a8c6-4578-9120-c9d44062bd68 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:52.786683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a7e3c-cc7d-418c-b3ab-e39511f3d989 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-8bcb7974-a8c6-4578-9120-c9d44062bd68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:52.793165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:52.794051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:52.794909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:52.794909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:52.794909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:52.794909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:52.794909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 254/516] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:52.795184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb7974-a8c6-4578-9120-c9d44062bd68 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:52.830566 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea67a5-a4a1-4c4c-ba21-f8d34dd23eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:52.831823 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea67a5-a4a1-4c4c-ba21-f8d34dd23eee None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:52.831823 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea67a5-a4a1-4c4c-ba21-f8d34dd23eee None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:52.832073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea67a5-a4a1-4c4c-ba21-f8d34dd23eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:52.842972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea67a5-a4a1-4c4c-ba21-f8d34dd23eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:52.844494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea67a5-a4a1-4c4c-ba21-f8d34dd23eee None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e4ea67a5-a4a1-4c4c-ba21-f8d34dd23eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:52.863129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:52.864110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:52.864951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:52.864951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:52.864951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:52.864951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:52.864951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 263/517] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:52.865215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea67a5-a4a1-4c4c-ba21-f8d34dd23eee None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:52.902755 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb7974-a8c6-4578-9120-c9d44062bd68 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:52.919248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:52.937704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e6991-d852-44f7-829a-db4a12e5843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:52.938888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e6991-d852-44f7-829a-db4a12e5843c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:52.938888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e6991-d852-44f7-829a-db4a12e5843c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:52.939054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e6991-d852-44f7-829a-db4a12e5843c None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:52.947121 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e6991-d852-44f7-829a-db4a12e5843c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:52.948472 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e6991-d852-44f7-829a-db4a12e5843c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-266e6991-d852-44f7-829a-db4a12e5843c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:39:52.963289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:52.964178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:52.965001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:39:52.965001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:39:52.965001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:39:52.965001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:39:52.965001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:39:52.965001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:39:52.965001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:39:52.965001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:52.965001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 255/518] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:52.965903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e6991-d852-44f7-829a-db4a12e5843c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:52.969372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea67a5-a4a1-4c4c-ba21-f8d34dd23eee None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:52.994584 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:53.005313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81829d46-6b2d-4d9e-a056-7f8e4af96eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:53.005981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81829d46-6b2d-4d9e-a056-7f8e4af96eb8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:53.013329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81829d46-6b2d-4d9e-a056-7f8e4af96eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:53.013329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81829d46-6b2d-4d9e-a056-7f8e4af96eb8 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:53.013329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81829d46-6b2d-4d9e-a056-7f8e4af96eb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:53.013329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81829d46-6b2d-4d9e-a056-7f8e4af96eb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:53.013329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81829d46-6b2d-4d9e-a056-7f8e4af96eb8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:53.020535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 264/519] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:53.021360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81829d46-6b2d-4d9e-a056-7f8e4af96eb8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:53.072678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e6991-d852-44f7-829a-db4a12e5843c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:53.107265 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ba51c-2e25-478c-b968-ce770048b2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:53.107996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ba51c-2e25-478c-b968-ce770048b2b4 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:53.108192 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ba51c-2e25-478c-b968-ce770048b2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:53.108469 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ba51c-2e25-478c-b968-ce770048b2b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:53.122029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81829d46-6b2d-4d9e-a056-7f8e4af96eb8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:53.136676 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:53.159547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf0ee2-08bd-407d-9aba-0d7fa075125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:53.160357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf0ee2-08bd-407d-9aba-0d7fa075125b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:53.160687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf0ee2-08bd-407d-9aba-0d7fa075125b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:53.161012 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf0ee2-08bd-407d-9aba-0d7fa075125b None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:53.162068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf0ee2-08bd-407d-9aba-0d7fa075125b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:53.162444 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf0ee2-08bd-407d-9aba-0d7fa075125b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:53.164096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf0ee2-08bd-407d-9aba-0d7fa075125b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:53.178971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ba51c-2e25-478c-b968-ce770048b2b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:53.180435 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ba51c-2e25-478c-b968-ce770048b2b4 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:53.180627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 265/520] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:53.181480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf0ee2-08bd-407d-9aba-0d7fa075125b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:53.207236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:53.282897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf0ee2-08bd-407d-9aba-0d7fa075125b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:53.283581 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 256/521] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:52 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:53.284938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ba51c-2e25-478c-b968-ce770048b2b4 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:53.324267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ffb12-d6ce-4ad4-805d-165e80d79bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:53.324930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ffb12-d6ce-4ad4-805d-165e80d79bee None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:53.325207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ffb12-d6ce-4ad4-805d-165e80d79bee None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:53.325476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ffb12-d6ce-4ad4-805d-165e80d79bee None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:53.326186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ffb12-d6ce-4ad4-805d-165e80d79bee None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:53.326480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ffb12-d6ce-4ad4-805d-165e80d79bee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:53.327885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ffb12-d6ce-4ad4-805d-165e80d79bee None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:53.339880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 266/522] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:53.340731 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ffb12-d6ce-4ad4-805d-165e80d79bee None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:53.373365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:53.389057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ba51c-2e25-478c-b968-ce770048b2b4 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:53.412391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:53.424281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09360e5-c85b-4100-bf68-8e8c2b88eecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:53.426070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09360e5-c85b-4100-bf68-8e8c2b88eecd None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:53.426348 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09360e5-c85b-4100-bf68-8e8c2b88eecd None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:53.426624 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09360e5-c85b-4100-bf68-8e8c2b88eecd None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:53.427458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09360e5-c85b-4100-bf68-8e8c2b88eecd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:53.427779 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09360e5-c85b-4100-bf68-8e8c2b88eecd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:53.429222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09360e5-c85b-4100-bf68-8e8c2b88eecd None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:53.447464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ffb12-d6ce-4ad4-805d-165e80d79bee None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:53.454144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 257/523] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:39:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:53.454969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09360e5-c85b-4100-bf68-8e8c2b88eecd None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:53.484898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a02e3-e812-4c69-8650-7eaffa942ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:53.485805 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a02e3-e812-4c69-8650-7eaffa942ad9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:53.486191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a02e3-e812-4c69-8650-7eaffa942ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:53.486549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a02e3-e812-4c69-8650-7eaffa942ad9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:53.494531 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a02e3-e812-4c69-8650-7eaffa942ad9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:53.496555 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a02e3-e812-4c69-8650-7eaffa942ad9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-143a02e3-e812-4c69-8650-7eaffa942ad9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:39:53.514003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:53.514914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:53.516076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:39:53.516076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:39:53.516076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:39:53.516076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:39:53.516076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:39:53.516076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:39:53.516076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:39:53.516076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:53.516076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 267/524] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:53.516706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a02e3-e812-4c69-8650-7eaffa942ad9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:53.560239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09360e5-c85b-4100-bf68-8e8c2b88eecd None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:53.594153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ec44d-a61c-46ac-aea5-127beb1a0e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:53.594886 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ec44d-a61c-46ac-aea5-127beb1a0e47 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:53.595167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ec44d-a61c-46ac-aea5-127beb1a0e47 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:53.595440 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ec44d-a61c-46ac-aea5-127beb1a0e47 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:53.597183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ec44d-a61c-46ac-aea5-127beb1a0e47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:53.598360 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ec44d-a61c-46ac-aea5-127beb1a0e47 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:53.609746 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:53.618262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a02e3-e812-4c69-8650-7eaffa942ad9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:53.650738 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcfae5-5954-42a9-aac3-5b980b7c4e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:53.651479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcfae5-5954-42a9-aac3-5b980b7c4e1f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:53.651757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcfae5-5954-42a9-aac3-5b980b7c4e1f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:53.652026 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcfae5-5954-42a9-aac3-5b980b7c4e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:53.662703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbcfae5-5954-42a9-aac3-5b980b7c4e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:53.664176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbcfae5-5954-42a9-aac3-5b980b7c4e1f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-4fbcfae5-5954-42a9-aac3-5b980b7c4e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:53.681260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:53.689029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:53.690142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:53.690142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:53.690142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:53.690142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:53.690142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 268/525] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:53.690142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcfae5-5954-42a9-aac3-5b980b7c4e1f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:53.714399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 258/526] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:39:53 2022] POST /identity/v3/users => generated 248 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:39:53.715468 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ec44d-a61c-46ac-aea5-127beb1a0e47 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:53.764162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:53.797890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcfae5-5954-42a9-aac3-5b980b7c4e1f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:53.840532 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ec44d-a61c-46ac-aea5-127beb1a0e47 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:53.841713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94771f-e0da-43d0-93cf-54ce7ef54a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:53.842529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94771f-e0da-43d0-93cf-54ce7ef54a3a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:53.842807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94771f-e0da-43d0-93cf-54ce7ef54a3a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:53.843107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94771f-e0da-43d0-93cf-54ce7ef54a3a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:53.851441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94771f-e0da-43d0-93cf-54ce7ef54a3a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:53.853212 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94771f-e0da-43d0-93cf-54ce7ef54a3a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-fd94771f-e0da-43d0-93cf-54ce7ef54a3a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:39:53.871750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:53.872868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:53.873759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:53.873759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:53.873759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:39:53.873759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:39:53.873759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:39:53.873759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:39:53.873759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:39:53.873759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:39:53.873759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:39:53.873759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:53.873759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 269/527] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:53.874391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94771f-e0da-43d0-93cf-54ce7ef54a3a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:53.882876 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:53.888434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d63eb-5396-4591-8dbd-062d93e88d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:53.889252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d63eb-5396-4591-8dbd-062d93e88d95 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:53.889556 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d63eb-5396-4591-8dbd-062d93e88d95 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:53.889795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d63eb-5396-4591-8dbd-062d93e88d95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:53.895130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d63eb-5396-4591-8dbd-062d93e88d95 None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=683c54f489f440b9842cb94980214df1) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:53.895130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d63eb-5396-4591-8dbd-062d93e88d95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:53.895130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d63eb-5396-4591-8dbd-062d93e88d95 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:53.912210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 259/528] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:53 2022] GET /identity/v3/role_assignments?user.id=683c54f489f440b9842cb94980214df1&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:53.912210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d63eb-5396-4591-8dbd-062d93e88d95 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:54.026364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94771f-e0da-43d0-93cf-54ce7ef54a3a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:54.035585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d63eb-5396-4591-8dbd-062d93e88d95 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:54.037350 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:54.075377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95155dae-0250-4703-83d8-0a51d3cdd5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:54.076335 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95155dae-0250-4703-83d8-0a51d3cdd5dc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:54.076712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95155dae-0250-4703-83d8-0a51d3cdd5dc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:54.077074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95155dae-0250-4703-83d8-0a51d3cdd5dc None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:54.077991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95155dae-0250-4703-83d8-0a51d3cdd5dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:54.078383 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95155dae-0250-4703-83d8-0a51d3cdd5dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:54.080272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95155dae-0250-4703-83d8-0a51d3cdd5dc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:54.080716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcc7c2-f956-4cc1-9ecd-94505570d74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:54.082003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcc7c2-f956-4cc1-9ecd-94505570d74a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:54.082674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcc7c2-f956-4cc1-9ecd-94505570d74a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:54.083124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcc7c2-f956-4cc1-9ecd-94505570d74a None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:54.083984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcc7c2-f956-4cc1-9ecd-94505570d74a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:54.084569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcc7c2-f956-4cc1-9ecd-94505570d74a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:54.086200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcc7c2-f956-4cc1-9ecd-94505570d74a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:54.105443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 260/529] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:54.105443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcc7c2-f956-4cc1-9ecd-94505570d74a None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:54.110888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 270/530] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:39:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:54.118918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95155dae-0250-4703-83d8-0a51d3cdd5dc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:54.187912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:54.229839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95155dae-0250-4703-83d8-0a51d3cdd5dc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:54.247003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcc7c2-f956-4cc1-9ecd-94505570d74a None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:54.277245 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed0a2d-e9c1-4a7e-8522-8d9d922e7fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:54.278303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed0a2d-e9c1-4a7e-8522-8d9d922e7fac None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:54.278585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed0a2d-e9c1-4a7e-8522-8d9d922e7fac None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:54.279312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed0a2d-e9c1-4a7e-8522-8d9d922e7fac None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:54.289721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed0a2d-e9c1-4a7e-8522-8d9d922e7fac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:54.290269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae95504-86ce-44af-aba8-e8998ab803c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:54.291093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae95504-86ce-44af-aba8-e8998ab803c0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:54.291607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed0a2d-e9c1-4a7e-8522-8d9d922e7fac None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:54.291807 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae95504-86ce-44af-aba8-e8998ab803c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:54.292175 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae95504-86ce-44af-aba8-e8998ab803c0 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:54.293109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae95504-86ce-44af-aba8-e8998ab803c0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:54.293534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae95504-86ce-44af-aba8-e8998ab803c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:54.295075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae95504-86ce-44af-aba8-e8998ab803c0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-3eed0a2d-e9c1-4a7e-8522-8d9d922e7fac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:39:54.310254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:54.311643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:54.312682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:39:54.312682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:39:54.312682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:39:54.312682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:39:54.312682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:39:54.312682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:39:54.312682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:39:54.312682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:54.317899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 271/531] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:54.317899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed0a2d-e9c1-4a7e-8522-8d9d922e7fac None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:54.320499 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 261/532] 10.222.0.121 () {56 vars in 1142 bytes} [Thu May 26 17:39:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:54.321484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae95504-86ce-44af-aba8-e8998ab803c0 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:54.346102 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:54.432273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae95504-86ce-44af-aba8-e8998ab803c0 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:54.460678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed0a2d-e9c1-4a7e-8522-8d9d922e7fac None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:54.470015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc399c42-6746-4f37-9b33-dbdc4c98266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:54.470986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc399c42-6746-4f37-9b33-dbdc4c98266e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:54.471658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc399c42-6746-4f37-9b33-dbdc4c98266e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:54.472429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc399c42-6746-4f37-9b33-dbdc4c98266e None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:54.473373 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc399c42-6746-4f37-9b33-dbdc4c98266e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:54.473674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc399c42-6746-4f37-9b33-dbdc4c98266e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:54.475212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc399c42-6746-4f37-9b33-dbdc4c98266e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:54.488189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 262/533] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:39:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:54.489054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc399c42-6746-4f37-9b33-dbdc4c98266e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:54.496440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:54.514667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a1d1b-bab6-4274-a97d-7b06f3697dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:54.515371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a1d1b-bab6-4274-a97d-7b06f3697dc5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:54.515655 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a1d1b-bab6-4274-a97d-7b06f3697dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:54.515993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a1d1b-bab6-4274-a97d-7b06f3697dc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:54.516877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a1d1b-bab6-4274-a97d-7b06f3697dc5 None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=c0889a69a018452d8de1924072739d17) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:54.517739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a1d1b-bab6-4274-a97d-7b06f3697dc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:54.520015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a1d1b-bab6-4274-a97d-7b06f3697dc5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:54.529189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 272/534] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:54 2022] GET /identity/v3/role_assignments?user.id=c0889a69a018452d8de1924072739d17&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:54.530029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a1d1b-bab6-4274-a97d-7b06f3697dc5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:54.587229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:54.608735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc399c42-6746-4f37-9b33-dbdc4c98266e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:54.649048 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240e257-fed7-47b5-abec-b955aa8b4da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:54.649866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240e257-fed7-47b5-abec-b955aa8b4da6 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:54.650243 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240e257-fed7-47b5-abec-b955aa8b4da6 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:54.650610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240e257-fed7-47b5-abec-b955aa8b4da6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:54.675354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a1d1b-bab6-4274-a97d-7b06f3697dc5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:54.749722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240e257-fed7-47b5-abec-b955aa8b4da6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:54.749722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240e257-fed7-47b5-abec-b955aa8b4da6 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:54.782667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c0d5d-83ba-4f83-ba7c-58bdb290c4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:54.783417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c0d5d-83ba-4f83-ba7c-58bdb290c4a5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:54.784230 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c0d5d-83ba-4f83-ba7c-58bdb290c4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:54.784744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c0d5d-83ba-4f83-ba7c-58bdb290c4a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:54.785396 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c0d5d-83ba-4f83-ba7c-58bdb290c4a5 None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=08709b462300437598c8db331e5ad20f) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:54.785704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c0d5d-83ba-4f83-ba7c-58bdb290c4a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:54.787195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c0d5d-83ba-4f83-ba7c-58bdb290c4a5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:54.805031 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 273/535] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:54 2022] GET /identity/v3/role_assignments?user.id=08709b462300437598c8db331e5ad20f&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:54.855089 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:54.878502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 263/536] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:54 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:54.887977 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c0d5d-83ba-4f83-ba7c-58bdb290c4a5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:54.975214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:55.050377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c0d5d-83ba-4f83-ba7c-58bdb290c4a5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:55.099658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aed933-3b9c-4d9d-ad3d-aaceb7a33c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:55.100493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aed933-3b9c-4d9d-ad3d-aaceb7a33c76 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:55.100757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aed933-3b9c-4d9d-ad3d-aaceb7a33c76 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:55.101022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aed933-3b9c-4d9d-ad3d-aaceb7a33c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:55.116798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aed933-3b9c-4d9d-ad3d-aaceb7a33c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:55.118302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aed933-3b9c-4d9d-ad3d-aaceb7a33c76 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-71aed933-3b9c-4d9d-ad3d-aaceb7a33c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:55.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:55.145244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:55.146207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:55.146207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:55.146207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:55.146207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:55.146207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 274/537] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:55.152965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240e257-fed7-47b5-abec-b955aa8b4da6 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:55.324392 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240e257-fed7-47b5-abec-b955aa8b4da6 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:55.376384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1acfe-3a4d-4c48-b8b4-b304eceec292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:55.377057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1acfe-3a4d-4c48-b8b4-b304eceec292 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:55.377335 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1acfe-3a4d-4c48-b8b4-b304eceec292 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:55.377806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1acfe-3a4d-4c48-b8b4-b304eceec292 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:55.378541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1acfe-3a4d-4c48-b8b4-b304eceec292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:55.378862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1acfe-3a4d-4c48-b8b4-b304eceec292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:55.380257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1acfe-3a4d-4c48-b8b4-b304eceec292 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:55.398602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 264/538] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:55.405893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aed933-3b9c-4d9d-ad3d-aaceb7a33c76 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:55.547654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aed933-3b9c-4d9d-ad3d-aaceb7a33c76 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:55.552360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:55.589611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b3010-2d43-41e7-a989-5a1bb04487a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:55.590337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b3010-2d43-41e7-a989-5a1bb04487a9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:55.590617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b3010-2d43-41e7-a989-5a1bb04487a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:55.591160 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b3010-2d43-41e7-a989-5a1bb04487a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:55.598870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b3010-2d43-41e7-a989-5a1bb04487a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:55.600466 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b3010-2d43-41e7-a989-5a1bb04487a9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-397b3010-2d43-41e7-a989-5a1bb04487a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:39:55.620743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:55.621673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:55.622499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:39:55.622499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:39:55.622499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:39:55.622499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:39:55.622499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:39:55.622499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:39:55.622499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:39:55.622499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:55.622499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 275/539] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:55.632423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1acfe-3a4d-4c48-b8b4-b304eceec292 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:55.723542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:55.807444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1acfe-3a4d-4c48-b8b4-b304eceec292 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:55.842802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec9cbc-8625-469f-acea-08b307923c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:55.843435 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec9cbc-8625-469f-acea-08b307923c6d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:55.843715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec9cbc-8625-469f-acea-08b307923c6d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:55.843988 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec9cbc-8625-469f-acea-08b307923c6d None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:55.844784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec9cbc-8625-469f-acea-08b307923c6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:55.845091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec9cbc-8625-469f-acea-08b307923c6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:55.846765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec9cbc-8625-469f-acea-08b307923c6d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:55.865341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 265/540] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:39:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:55.872257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b3010-2d43-41e7-a989-5a1bb04487a9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:56.061124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b3010-2d43-41e7-a989-5a1bb04487a9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:56.116656 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356961e0-901e-45ee-a07f-8579f4213ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:56.117324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356961e0-901e-45ee-a07f-8579f4213ff0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:56.117602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356961e0-901e-45ee-a07f-8579f4213ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:56.117870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356961e0-901e-45ee-a07f-8579f4213ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:56.118848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356961e0-901e-45ee-a07f-8579f4213ff0 None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=25d85d02abed40199dd2515ecf84b7fb) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:56.119037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356961e0-901e-45ee-a07f-8579f4213ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:56.120693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356961e0-901e-45ee-a07f-8579f4213ff0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:56.131225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 276/541] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:39:55 2022] GET /identity/v3/role_assignments?user.id=25d85d02abed40199dd2515ecf84b7fb&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:56.427839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af2635-6a2c-4bf4-9357-f9af4f30f96a None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:56.428122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af2635-6a2c-4bf4-9357-f9af4f30f96a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:56.428671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af2635-6a2c-4bf4-9357-f9af4f30f96a None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:56.430508 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 266/542] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:56 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:56.440805 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963f090-f0c3-4c8a-bca4-2f60d81ab757 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:56.441416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963f090-f0c3-4c8a-bca4-2f60d81ab757 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:56.441909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963f090-f0c3-4c8a-bca4-2f60d81ab757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:56.463939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8963f090-f0c3-4c8a-bca4-2f60d81ab757 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:56.465540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8963f090-f0c3-4c8a-bca4-2f60d81ab757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:56.499577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:56.545990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 277/543] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:56.570549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af2635-6a2c-4bf4-9357-f9af4f30f96a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:56.702004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d09aa-f00e-4577-bdf5-ebf35cc9ec63 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:56.702684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d09aa-f00e-4577-bdf5-ebf35cc9ec63 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:56.703211 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d09aa-f00e-4577-bdf5-ebf35cc9ec63 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:56.705058 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 278/544] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:56 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:39:56.705644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:56.710732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af2635-6a2c-4bf4-9357-f9af4f30f96a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:56.712432 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6990e43-56ee-4d17-9fb9-bc5c7795f721 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:56.712722 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6990e43-56ee-4d17-9fb9-bc5c7795f721 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:56.712994 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6990e43-56ee-4d17-9fb9-bc5c7795f721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:56.745480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c6990e43-56ee-4d17-9fb9-bc5c7795f721 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:56.747667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c6990e43-56ee-4d17-9fb9-bc5c7795f721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:56.769143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63232717-b88b-4e80-8746-155cc0a38c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:56.770369 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63232717-b88b-4e80-8746-155cc0a38c18 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:56.770645 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63232717-b88b-4e80-8746-155cc0a38c18 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:56.770950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63232717-b88b-4e80-8746-155cc0a38c18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:56.792708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63232717-b88b-4e80-8746-155cc0a38c18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-63232717-b88b-4e80-8746-155cc0a38c18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:56.806011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:56.807098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:56.807925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 267/545] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:39:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:56.813739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63232717-b88b-4e80-8746-155cc0a38c18 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:56.896933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 279/546] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:56.935102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6990e43-56ee-4d17-9fb9-bc5c7795f721 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:56.991610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63232717-b88b-4e80-8746-155cc0a38c18 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:57.059030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3ecf7-62d0-4c6c-9d53-603598020c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:57.060138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3ecf7-62d0-4c6c-9d53-603598020c96 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:57.060633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3ecf7-62d0-4c6c-9d53-603598020c96 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:57.061124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3ecf7-62d0-4c6c-9d53-603598020c96 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:57.062087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3ecf7-62d0-4c6c-9d53-603598020c96 None admin] RBAC: Adding query filter params (name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:57.064083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3ecf7-62d0-4c6c-9d53-603598020c96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:57.066199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3ecf7-62d0-4c6c-9d53-603598020c96 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:57.072524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:57.082403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 268/547] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:39:56 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:57.086778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3ecf7-62d0-4c6c-9d53-603598020c96 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:57.109716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6990e43-56ee-4d17-9fb9-bc5c7795f721 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:57.145895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:57.175765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50cba4-ec68-4a14-b84b-79557bf00b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:57.177029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50cba4-ec68-4a14-b84b-79557bf00b74 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:57.177547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50cba4-ec68-4a14-b84b-79557bf00b74 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:57.178039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50cba4-ec68-4a14-b84b-79557bf00b74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:57.208263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50cba4-ec68-4a14-b84b-79557bf00b74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-ea50cba4-ec68-4a14-b84b-79557bf00b74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:57.216644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:57.217583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:57.218910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:57.218910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:57.218910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:57.218910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:57.218910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:57.218910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:57.218910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:57.218910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:57.218910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:57.218910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:57.218910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:57.218910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:57.231125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 280/548] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:39:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:39:57.244734 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50cba4-ec68-4a14-b84b-79557bf00b74 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:57.292588 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3ecf7-62d0-4c6c-9d53-603598020c96 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:57.337318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a51cc2-1209-46ed-be16-141f86e330fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:57.338508 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a51cc2-1209-46ed-be16-141f86e330fa None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:57.344073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a51cc2-1209-46ed-be16-141f86e330fa None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:57.344624 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a51cc2-1209-46ed-be16-141f86e330fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:57.365125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a51cc2-1209-46ed-be16-141f86e330fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:57.367842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a51cc2-1209-46ed-be16-141f86e330fa None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-22a51cc2-1209-46ed-be16-141f86e330fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:57.396785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:57.397756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:57.398640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:57.398640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:57.398640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:57.398640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:57.402551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 269/549] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:57.403931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8915571-671f-4f36-afd7-67ddb2133bca None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:57.404215 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8915571-671f-4f36-afd7-67ddb2133bca None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:57.404493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8915571-671f-4f36-afd7-67ddb2133bca None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:57.405783 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 270/550] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:57.406467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8915571-671f-4f36-afd7-67ddb2133bca None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:57.448133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50cba4-ec68-4a14-b84b-79557bf00b74 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:57.496748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:57.499260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35da92-5006-427a-a80f-0d4152899fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:57.500248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35da92-5006-427a-a80f-0d4152899fc3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:57.500772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35da92-5006-427a-a80f-0d4152899fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:57.501284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35da92-5006-427a-a80f-0d4152899fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:57.502277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35da92-5006-427a-a80f-0d4152899fc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:57.503410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35da92-5006-427a-a80f-0d4152899fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:57.506775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35da92-5006-427a-a80f-0d4152899fc3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:57.519186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 281/551] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:39:57 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:39:57.520961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686d66b-f0b1-4986-aacc-790c5a4f964a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:57.521487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686d66b-f0b1-4986-aacc-790c5a4f964a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:57.521977 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686d66b-f0b1-4986-aacc-790c5a4f964a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:57.528255 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8915571-671f-4f36-afd7-67ddb2133bca None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:57.543009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2686d66b-f0b1-4986-aacc-790c5a4f964a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:57.544594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2686d66b-f0b1-4986-aacc-790c5a4f964a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:57.548860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:57.573958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1c403-0c79-4a12-b771-2c83202fd2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:57.576001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1c403-0c79-4a12-b771-2c83202fd2b7 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:57.576001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1c403-0c79-4a12-b771-2c83202fd2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:57.576001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1c403-0c79-4a12-b771-2c83202fd2b7 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:57.576445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1c403-0c79-4a12-b771-2c83202fd2b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:57.576937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1c403-0c79-4a12-b771-2c83202fd2b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:57.578589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1c403-0c79-4a12-b771-2c83202fd2b7 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:57.595252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 271/552] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:57.597304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1c403-0c79-4a12-b771-2c83202fd2b7 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:57.635272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 282/553] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:57.636437 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686d66b-f0b1-4986-aacc-790c5a4f964a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:57.724891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1c403-0c79-4a12-b771-2c83202fd2b7 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:57.760529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686d66b-f0b1-4986-aacc-790c5a4f964a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:57.772650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b326dd7-6624-4749-ac25-bf9f3b95a33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:57.773572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b326dd7-6624-4749-ac25-bf9f3b95a33c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:57.773924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b326dd7-6624-4749-ac25-bf9f3b95a33c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:57.774269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b326dd7-6624-4749-ac25-bf9f3b95a33c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:57.784473 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:57.789534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b326dd7-6624-4749-ac25-bf9f3b95a33c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:57.791582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b326dd7-6624-4749-ac25-bf9f3b95a33c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:57.806978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a93bc3-e729-427d-800c-834cb9d4a23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:57.808482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a93bc3-e729-427d-800c-834cb9d4a23e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:57.809098 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a93bc3-e729-427d-800c-834cb9d4a23e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:57.809587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a93bc3-e729-427d-800c-834cb9d4a23e None admin] PATH_INFO: `/v3/users/swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-1b326dd7-6624-4749-ac25-bf9f3b95a33c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:57.816542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:57.817595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:57.818746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:57.818746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:57.818746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:57.818746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:57.822844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 272/554] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:57.823788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b326dd7-6624-4749-ac25-bf9f3b95a33c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:57.824056 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a93bc3-e729-427d-800c-834cb9d4a23e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:57.825875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a93bc3-e729-427d-800c-834cb9d4a23e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:57.831619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-48a93bc3-e729-427d-800c-834cb9d4a23e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:39:57.845140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:57.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:57.847602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:57.847602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:57.847602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:39:57.847602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:39:57.847602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:39:57.847602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:39:57.847602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:39:57.847602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:39:57.847602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 17:39:57.847602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:57.850093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 283/555] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:39:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:57.851152 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a93bc3-e729-427d-800c-834cb9d4a23e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:57.948795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b326dd7-6624-4749-ac25-bf9f3b95a33c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:57.965059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a93bc3-e729-427d-800c-834cb9d4a23e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:57.987724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44092f05-56be-44bf-9a0c-dd6a696c829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:57.988629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44092f05-56be-44bf-9a0c-dd6a696c829d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:57.988981 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44092f05-56be-44bf-9a0c-dd6a696c829d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:57.989683 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44092f05-56be-44bf-9a0c-dd6a696c829d None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:58.000180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44092f05-56be-44bf-9a0c-dd6a696c829d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:58.005098 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b0126-2b0d-4ed9-ba98-d10d6a53d773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:58.005766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b0126-2b0d-4ed9-ba98-d10d6a53d773 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:58.006051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b0126-2b0d-4ed9-ba98-d10d6a53d773 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:58.006327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b0126-2b0d-4ed9-ba98-d10d6a53d773 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:58.007080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b0126-2b0d-4ed9-ba98-d10d6a53d773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:58.007386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b0126-2b0d-4ed9-ba98-d10d6a53d773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-44092f05-56be-44bf-9a0c-dd6a696c829d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:58.008668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:58.009561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:58.011501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:58.011501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:58.011501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:58.011501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:58.011501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:58.011501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:58.011501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:58.011501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:58.011501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:58.011501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:58.011501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:58.011501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:58.012477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b0126-2b0d-4ed9-ba98-d10d6a53d773 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:58.012621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 273/556] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:39:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:58.012621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44092f05-56be-44bf-9a0c-dd6a696c829d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:58.023845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 284/557] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:58.025231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a141c8a-5061-47ac-bcac-8c57985b10da None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:58.025520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a141c8a-5061-47ac-bcac-8c57985b10da None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:58.025799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a141c8a-5061-47ac-bcac-8c57985b10da None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:58.030864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 285/558] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:58.030864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a141c8a-5061-47ac-bcac-8c57985b10da None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:58.084924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:58.106809 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:58.147740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44092f05-56be-44bf-9a0c-dd6a696c829d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:58.154266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a141c8a-5061-47ac-bcac-8c57985b10da None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:58.187446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d55afe-e28a-4bc6-9ec7-39d1b347962b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:58.188121 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d55afe-e28a-4bc6-9ec7-39d1b347962b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:58.188395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d55afe-e28a-4bc6-9ec7-39d1b347962b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:58.188663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d55afe-e28a-4bc6-9ec7-39d1b347962b None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:58.189490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d55afe-e28a-4bc6-9ec7-39d1b347962b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:58.190634 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d55afe-e28a-4bc6-9ec7-39d1b347962b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:58.192365 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d55afe-e28a-4bc6-9ec7-39d1b347962b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:58.195441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b5e21-5266-4c09-90be-9a5b7bf62216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:58.196107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b5e21-5266-4c09-90be-9a5b7bf62216 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:58.196382 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b5e21-5266-4c09-90be-9a5b7bf62216 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:58.196651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b5e21-5266-4c09-90be-9a5b7bf62216 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:58.197397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b5e21-5266-4c09-90be-9a5b7bf62216 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:58.197696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b5e21-5266-4c09-90be-9a5b7bf62216 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:58.199114 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b5e21-5266-4c09-90be-9a5b7bf62216 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:58.212381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 286/559] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:39:58 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:58.212381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b5e21-5266-4c09-90be-9a5b7bf62216 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:58.216785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 274/560] 10.222.0.121 () {56 vars in 1140 bytes} [Thu May 26 17:39:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:58.221304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac1be0-4e67-424b-96ee-35fe8afd4806 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:58.221304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac1be0-4e67-424b-96ee-35fe8afd4806 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:58.221304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac1be0-4e67-424b-96ee-35fe8afd4806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:58.237862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c2ac1be0-4e67-424b-96ee-35fe8afd4806 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:58.239209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c2ac1be0-4e67-424b-96ee-35fe8afd4806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:58.326933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 275/561] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:58.326933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac1be0-4e67-424b-96ee-35fe8afd4806 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:58.332715 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b5e21-5266-4c09-90be-9a5b7bf62216 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:58.361398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:58.364618 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:58.373959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b94a34-5e57-42b6-8f61-8578b0d0c9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:58.375067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b94a34-5e57-42b6-8f61-8578b0d0c9bf None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:58.375166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b94a34-5e57-42b6-8f61-8578b0d0c9bf None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:58.375436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b94a34-5e57-42b6-8f61-8578b0d0c9bf None admin] PATH_INFO: `/v3/users/glance` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:58.383888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b94a34-5e57-42b6-8f61-8578b0d0c9bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:58.385561 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b94a34-5e57-42b6-8f61-8578b0d0c9bf None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e8b94a34-5e57-42b6-8f61-8578b0d0c9bf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:39:58.401877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:58.402913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:58.403936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:58.403936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:58.403936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:39:58.403936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:39:58.403936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:39:58.403936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:39:58.403936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:39:58.403936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:39:58.403936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 17:39:58.403936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:58.411974 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 287/562] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:39:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:58.411974 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b94a34-5e57-42b6-8f61-8578b0d0c9bf None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:58.439548 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac1be0-4e67-424b-96ee-35fe8afd4806 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:58.482678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05115f25-bb73-448e-86bf-665e882c7449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:58.483533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05115f25-bb73-448e-86bf-665e882c7449 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:58.484047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05115f25-bb73-448e-86bf-665e882c7449 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:58.484324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05115f25-bb73-448e-86bf-665e882c7449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:58.496467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05115f25-bb73-448e-86bf-665e882c7449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:58.498204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05115f25-bb73-448e-86bf-665e882c7449 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-05115f25-bb73-448e-86bf-665e882c7449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:58.519944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:58.520920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:58.521806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:58.521806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:58.521806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:58.521806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:58.521806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 276/563] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:58.522098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05115f25-bb73-448e-86bf-665e882c7449 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:58.529112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b94a34-5e57-42b6-8f61-8578b0d0c9bf None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:58.567673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9bc40-05d1-4f72-b0fb-40343902ce22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:58.568326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9bc40-05d1-4f72-b0fb-40343902ce22 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:58.568605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9bc40-05d1-4f72-b0fb-40343902ce22 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:58.568876 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9bc40-05d1-4f72-b0fb-40343902ce22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:58.624771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:58.630914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:58.641509 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05115f25-bb73-448e-86bf-665e882c7449 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:58.651058 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9bc40-05d1-4f72-b0fb-40343902ce22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:58.652536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9bc40-05d1-4f72-b0fb-40343902ce22 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:58.679350 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecadc7-6d62-440d-a8b7-18b551b91800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:58.680154 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecadc7-6d62-440d-a8b7-18b551b91800 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:58.680430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecadc7-6d62-440d-a8b7-18b551b91800 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:58.680698 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecadc7-6d62-440d-a8b7-18b551b91800 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:58.692354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecadc7-6d62-440d-a8b7-18b551b91800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-5eecadc7-6d62-440d-a8b7-18b551b91800 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:58.700476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:58.701423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:58.702269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 277/564] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:39:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:58.703010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecadc7-6d62-440d-a8b7-18b551b91800 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:58.768741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 288/565] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:39:58 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:39:58.769728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9bc40-05d1-4f72-b0fb-40343902ce22 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:58.828379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecadc7-6d62-440d-a8b7-18b551b91800 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:58.863774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30cc60-0454-4e3f-8091-d24ee2bf4f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:58.864444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30cc60-0454-4e3f-8091-d24ee2bf4f12 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:58.864720 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30cc60-0454-4e3f-8091-d24ee2bf4f12 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:58.864989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30cc60-0454-4e3f-8091-d24ee2bf4f12 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:58.865817 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30cc60-0454-4e3f-8091-d24ee2bf4f12 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:58.866127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30cc60-0454-4e3f-8091-d24ee2bf4f12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:58.867590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30cc60-0454-4e3f-8091-d24ee2bf4f12 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:58.879056 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:58.885759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 278/566] 10.222.0.121 () {56 vars in 1142 bytes} [Thu May 26 17:39:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:58.886583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30cc60-0454-4e3f-8091-d24ee2bf4f12 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:58.890149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:58.899460 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9bc40-05d1-4f72-b0fb-40343902ce22 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:58.934771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65ae7c-ec4b-4f4c-a9f4-331363b952db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:58.935467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65ae7c-ec4b-4f4c-a9f4-331363b952db None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:58.935750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65ae7c-ec4b-4f4c-a9f4-331363b952db None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:58.936027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65ae7c-ec4b-4f4c-a9f4-331363b952db None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:58.936746 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65ae7c-ec4b-4f4c-a9f4-331363b952db None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:58.937055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65ae7c-ec4b-4f4c-a9f4-331363b952db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:58.938449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65ae7c-ec4b-4f4c-a9f4-331363b952db None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:58.950019 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 289/567] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:58.950920 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65ae7c-ec4b-4f4c-a9f4-331363b952db None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:58.991078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30cc60-0454-4e3f-8091-d24ee2bf4f12 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:59.025358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33e368-b031-4915-9b6c-4cb023bee5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:59.026053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33e368-b031-4915-9b6c-4cb023bee5b5 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.026278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33e368-b031-4915-9b6c-4cb023bee5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.026544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33e368-b031-4915-9b6c-4cb023bee5b5 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.027328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33e368-b031-4915-9b6c-4cb023bee5b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:59.027630 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33e368-b031-4915-9b6c-4cb023bee5b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:59.028973 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33e368-b031-4915-9b6c-4cb023bee5b5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:59.037978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 279/568] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:39:58 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:59.039269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c0fcf-ab63-4ae3-b78f-75f0acc1721f None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.039553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c0fcf-ab63-4ae3-b78f-75f0acc1721f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.039825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c0fcf-ab63-4ae3-b78f-75f0acc1721f None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.041043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 280/569] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:39:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:39:59.041979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c0fcf-ab63-4ae3-b78f-75f0acc1721f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:59.055428 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65ae7c-ec4b-4f4c-a9f4-331363b952db None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:59.091082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a4458-7c0f-465d-93f9-04dfb935b1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:59.091890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a4458-7c0f-465d-93f9-04dfb935b1e9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.092177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a4458-7c0f-465d-93f9-04dfb935b1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.092456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a4458-7c0f-465d-93f9-04dfb935b1e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.104694 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a4458-7c0f-465d-93f9-04dfb935b1e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:59.106370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a4458-7c0f-465d-93f9-04dfb935b1e9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:59.108092 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:59.113387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-631a4458-7c0f-465d-93f9-04dfb935b1e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:59.124435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:59.125447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:59.126331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:59.126331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:59.126331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:59.126331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:59.126331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 290/570] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:59.126624 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a4458-7c0f-465d-93f9-04dfb935b1e9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:59.152361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c0fcf-ab63-4ae3-b78f-75f0acc1721f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:59.186752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea198ae-1528-436f-bea3-4ecb49d5b1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:59.187573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea198ae-1528-436f-bea3-4ecb49d5b1e6 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.187853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea198ae-1528-436f-bea3-4ecb49d5b1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.188123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea198ae-1528-436f-bea3-4ecb49d5b1e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.198903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea198ae-1528-436f-bea3-4ecb49d5b1e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:59.200360 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea198ae-1528-436f-bea3-4ecb49d5b1e6 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-eea198ae-1528-436f-bea3-4ecb49d5b1e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:59.218176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:59.219443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:59.220326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:59.220326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:59.220326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:59.220326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:59.220326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 281/571] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:59.225603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea198ae-1528-436f-bea3-4ecb49d5b1e6 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:59.229575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a4458-7c0f-465d-93f9-04dfb935b1e9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:59.263822 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10949cd2-b5d1-48a4-937e-39065d6c3f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:59.264778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10949cd2-b5d1-48a4-937e-39065d6c3f81 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.265078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10949cd2-b5d1-48a4-937e-39065d6c3f81 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.265369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10949cd2-b5d1-48a4-937e-39065d6c3f81 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.273932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10949cd2-b5d1-48a4-937e-39065d6c3f81 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:59.275404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10949cd2-b5d1-48a4-937e-39065d6c3f81 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-10949cd2-b5d1-48a4-937e-39065d6c3f81 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:39:59.289133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:59.290125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:59.291134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:59.291134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:59.291134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:39:59.291134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:39:59.291134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:39:59.291134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:39:59.291134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:39:59.291134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:39:59.291134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:39:59.291134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:39:59.291134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 291/572] 10.222.0.121 () {56 vars in 1098 bytes} [Thu May 26 17:39:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:59.292543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7a452-7d5d-41e5-90eb-5c2bc6a7e16d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.292543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7a452-7d5d-41e5-90eb-5c2bc6a7e16d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.292543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7a452-7d5d-41e5-90eb-5c2bc6a7e16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.308521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a2a7a452-7d5d-41e5-90eb-5c2bc6a7e16d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:39:59.309717 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a2a7a452-7d5d-41e5-90eb-5c2bc6a7e16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:39:59.331937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea198ae-1528-436f-bea3-4ecb49d5b1e6 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:59.356132 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:59.373832 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:59.375075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28200f-0ac3-42b4-8e97-28e607a2f31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:59.375877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28200f-0ac3-42b4-8e97-28e607a2f31b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.376161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28200f-0ac3-42b4-8e97-28e607a2f31b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.376433 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28200f-0ac3-42b4-8e97-28e607a2f31b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.387539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28200f-0ac3-42b4-8e97-28e607a2f31b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:59.389179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28200f-0ac3-42b4-8e97-28e607a2f31b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:59.391492 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 292/573] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:39:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:39:59.392539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7a452-7d5d-41e5-90eb-5c2bc6a7e16d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-df28200f-0ac3-42b4-8e97-28e607a2f31b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:39:59.414953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:59.416558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:59.417452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:39:59.417452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:39:59.417452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:39:59.417452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:59.417452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 282/574] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:59.417751 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28200f-0ac3-42b4-8e97-28e607a2f31b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:59.503364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7a452-7d5d-41e5-90eb-5c2bc6a7e16d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:59.520507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28200f-0ac3-42b4-8e97-28e607a2f31b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:59.536852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec488b0b-5b51-4db7-a732-781f5207353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:59.537523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec488b0b-5b51-4db7-a732-781f5207353d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.537802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec488b0b-5b51-4db7-a732-781f5207353d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.538072 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec488b0b-5b51-4db7-a732-781f5207353d None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.538839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec488b0b-5b51-4db7-a732-781f5207353d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:59.539147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec488b0b-5b51-4db7-a732-781f5207353d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:59.540543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec488b0b-5b51-4db7-a732-781f5207353d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:59.552513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 293/575] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:59.552513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec488b0b-5b51-4db7-a732-781f5207353d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:59.558889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8fc06-a1df-4c12-8587-a52bba071936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:59.559638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8fc06-a1df-4c12-8587-a52bba071936 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.559915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8fc06-a1df-4c12-8587-a52bba071936 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.560187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8fc06-a1df-4c12-8587-a52bba071936 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.560924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8fc06-a1df-4c12-8587-a52bba071936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:59.561290 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8fc06-a1df-4c12-8587-a52bba071936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:59.562664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8fc06-a1df-4c12-8587-a52bba071936 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:59.576062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 283/576] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:59.576697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8fc06-a1df-4c12-8587-a52bba071936 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:59.605469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:59.607561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:59.659190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec488b0b-5b51-4db7-a732-781f5207353d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:59.680255 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8fc06-a1df-4c12-8587-a52bba071936 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:59.694623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04214901-035b-4f21-9945-31bcac9f0c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:59.695326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04214901-035b-4f21-9945-31bcac9f0c49 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.695632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04214901-035b-4f21-9945-31bcac9f0c49 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.695910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04214901-035b-4f21-9945-31bcac9f0c49 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.696748 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04214901-035b-4f21-9945-31bcac9f0c49 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:59.697059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04214901-035b-4f21-9945-31bcac9f0c49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:59.698495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04214901-035b-4f21-9945-31bcac9f0c49 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:59.716050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 294/577] 10.222.0.121 () {56 vars in 1138 bytes} [Thu May 26 17:39:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:59.716876 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04214901-035b-4f21-9945-31bcac9f0c49 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:59.717027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7e2e3-127c-48d3-8f70-9c3eb2ebdf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:59.717751 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7e2e3-127c-48d3-8f70-9c3eb2ebdf38 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.718127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7e2e3-127c-48d3-8f70-9c3eb2ebdf38 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.718406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7e2e3-127c-48d3-8f70-9c3eb2ebdf38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.729111 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7e2e3-127c-48d3-8f70-9c3eb2ebdf38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-34b7e2e3-127c-48d3-8f70-9c3eb2ebdf38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:39:59.737660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:59.740563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 284/578] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:39:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:59.741561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7e2e3-127c-48d3-8f70-9c3eb2ebdf38 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:59.825456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04214901-035b-4f21-9945-31bcac9f0c49 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:59.827045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:59.835705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:39:59.844579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7e2e3-127c-48d3-8f70-9c3eb2ebdf38 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:39:59.860641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b2896-b5e0-4328-916c-b8869cb244c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:59.862231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b2896-b5e0-4328-916c-b8869cb244c4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.862750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b2896-b5e0-4328-916c-b8869cb244c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.863288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b2896-b5e0-4328-916c-b8869cb244c4 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.864307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b2896-b5e0-4328-916c-b8869cb244c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:39:59.864685 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b2896-b5e0-4328-916c-b8869cb244c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:59.866169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b2896-b5e0-4328-916c-b8869cb244c4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:59.878515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 295/579] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:39:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:39:59.879626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b2896-b5e0-4328-916c-b8869cb244c4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:59.881823 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca21c8-06cb-400f-8c05-efbb7bbf4dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:39:59.882633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca21c8-06cb-400f-8c05-efbb7bbf4dc9 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:39:59.882946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca21c8-06cb-400f-8c05-efbb7bbf4dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:39:59.883527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca21c8-06cb-400f-8c05-efbb7bbf4dc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:39:59.891591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca21c8-06cb-400f-8c05-efbb7bbf4dc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:39:59.893266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca21c8-06cb-400f-8c05-efbb7bbf4dc9 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-b7ca21c8-06cb-400f-8c05-efbb7bbf4dc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:39:59.907543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:39:59.909118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:39:59.910178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:39:59.910178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:39:59.910178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:39:59.910178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:39:59.910178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:39:59.910178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:39:59.910178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:39:59.910178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:39:59.911235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 285/580] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:39:59.911527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca21c8-06cb-400f-8c05-efbb7bbf4dc9 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:39:59.984446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b2896-b5e0-4328-916c-b8869cb244c4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:00.014956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca21c8-06cb-400f-8c05-efbb7bbf4dc9 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:00.019617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d65127-879c-442b-9e4c-6dd4d7ffc306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:00.020551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d65127-879c-442b-9e4c-6dd4d7ffc306 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:00.020835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d65127-879c-442b-9e4c-6dd4d7ffc306 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:00.021109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d65127-879c-442b-9e4c-6dd4d7ffc306 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:00.027728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d65127-879c-442b-9e4c-6dd4d7ffc306 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:00.029161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d65127-879c-442b-9e4c-6dd4d7ffc306 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-25d65127-879c-442b-9e4c-6dd4d7ffc306 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:00.044342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:00.045461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:00.046521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:00.046521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:00.046521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:00.046521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:00.046521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:00.046521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:00.046521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:00.046521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:00.046521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 296/581] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:39:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:00.047145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d65127-879c-442b-9e4c-6dd4d7ffc306 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:00.056359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14ef90-301a-4f59-a65c-255117394741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:00.057194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14ef90-301a-4f59-a65c-255117394741 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:00.057482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14ef90-301a-4f59-a65c-255117394741 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:00.057772 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14ef90-301a-4f59-a65c-255117394741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:00.070189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14ef90-301a-4f59-a65c-255117394741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:00.071869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14ef90-301a-4f59-a65c-255117394741 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:00.072054 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-be14ef90-301a-4f59-a65c-255117394741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:00.089741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:00.090904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:00.091864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:00.091864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:00.091864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:00.091864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:00.095112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 286/582] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:39:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:00.096443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14ef90-301a-4f59-a65c-255117394741 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:00.099902 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:00.153466 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d65127-879c-442b-9e4c-6dd4d7ffc306 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:00.189095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b1769-3f5c-49be-9242-9f204ee2c3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:00.190153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b1769-3f5c-49be-9242-9f204ee2c3e0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:00.190603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b1769-3f5c-49be-9242-9f204ee2c3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:00.191137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b1769-3f5c-49be-9242-9f204ee2c3e0 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:00.192358 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b1769-3f5c-49be-9242-9f204ee2c3e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:00.192888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b1769-3f5c-49be-9242-9f204ee2c3e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:00.195081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b1769-3f5c-49be-9242-9f204ee2c3e0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:00.203504 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 297/583] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:40:00 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:00.204323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b1769-3f5c-49be-9242-9f204ee2c3e0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:00.211921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14ef90-301a-4f59-a65c-255117394741 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:00.247995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d862f-4276-4ca5-8bcc-ed10e641d7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:00.248713 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d862f-4276-4ca5-8bcc-ed10e641d7bf None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:00.248995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d862f-4276-4ca5-8bcc-ed10e641d7bf None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:00.249272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d862f-4276-4ca5-8bcc-ed10e641d7bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:00.257328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d862f-4276-4ca5-8bcc-ed10e641d7bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:00.258759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d862f-4276-4ca5-8bcc-ed10e641d7bf None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-e32d862f-4276-4ca5-8bcc-ed10e641d7bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:00.274137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:00.275314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:00.276266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:00.276266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:00.276266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:00.276266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:00.276266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:00.276266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:00.276266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:00.276266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:00.276266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 17:40:00.276266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:00.276266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 287/584] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:40:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:00.276855 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d862f-4276-4ca5-8bcc-ed10e641d7bf None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:00.291887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:00.307099 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b1769-3f5c-49be-9242-9f204ee2c3e0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:00.340118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:00.342323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f5b03-c7ec-4c0d-b7b1-bfa8e5c5c84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:00.343029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f5b03-c7ec-4c0d-b7b1-bfa8e5c5c84e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:00.343309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f5b03-c7ec-4c0d-b7b1-bfa8e5c5c84e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:00.343588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f5b03-c7ec-4c0d-b7b1-bfa8e5c5c84e None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:00.344365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f5b03-c7ec-4c0d-b7b1-bfa8e5c5c84e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:00.344668 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f5b03-c7ec-4c0d-b7b1-bfa8e5c5c84e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:00.346801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f5b03-c7ec-4c0d-b7b1-bfa8e5c5c84e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:00.358487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 298/585] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:00.362649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f5b03-c7ec-4c0d-b7b1-bfa8e5c5c84e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:00.381682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d862f-4276-4ca5-8bcc-ed10e641d7bf None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:00.417846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0075dd-a4b5-4fb5-863f-34550f3e5c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:00.418514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0075dd-a4b5-4fb5-863f-34550f3e5c09 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:00.418798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0075dd-a4b5-4fb5-863f-34550f3e5c09 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:00.419106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0075dd-a4b5-4fb5-863f-34550f3e5c09 None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:00.419948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0075dd-a4b5-4fb5-863f-34550f3e5c09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:00.420261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0075dd-a4b5-4fb5-863f-34550f3e5c09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:00.421690 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0075dd-a4b5-4fb5-863f-34550f3e5c09 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:00.433995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 288/586] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:00.434839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0075dd-a4b5-4fb5-863f-34550f3e5c09 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:00.461906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f5b03-c7ec-4c0d-b7b1-bfa8e5c5c84e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:00.498087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68364af-0351-47d1-afe5-d20ffa6c592e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:00.498488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68364af-0351-47d1-afe5-d20ffa6c592e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:00.498770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68364af-0351-47d1-afe5-d20ffa6c592e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:00.499071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68364af-0351-47d1-afe5-d20ffa6c592e None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:00.499789 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68364af-0351-47d1-afe5-d20ffa6c592e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:00.500086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68364af-0351-47d1-afe5-d20ffa6c592e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:00.501463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68364af-0351-47d1-afe5-d20ffa6c592e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:00.513589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 299/587] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:00.514442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68364af-0351-47d1-afe5-d20ffa6c592e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:00.519191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:00.541136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0075dd-a4b5-4fb5-863f-34550f3e5c09 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:00.564286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:00.576460 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41398893-c03f-47a8-8ce4-5c48f4983ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:00.577257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41398893-c03f-47a8-8ce4-5c48f4983ce8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:00.577533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41398893-c03f-47a8-8ce4-5c48f4983ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:00.577802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41398893-c03f-47a8-8ce4-5c48f4983ce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:00.589599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41398893-c03f-47a8-8ce4-5c48f4983ce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:00.591096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41398893-c03f-47a8-8ce4-5c48f4983ce8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-41398893-c03f-47a8-8ce4-5c48f4983ce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:00.609716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:00.610788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:00.611985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:00.611985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:00.611985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:00.611985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:00.611985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 289/588] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:00.611985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41398893-c03f-47a8-8ce4-5c48f4983ce8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:00.621501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68364af-0351-47d1-afe5-d20ffa6c592e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:00.656837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da714b0b-c511-4ef5-8b46-531383699898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:00.657502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da714b0b-c511-4ef5-8b46-531383699898 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:00.657807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da714b0b-c511-4ef5-8b46-531383699898 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:00.658080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da714b0b-c511-4ef5-8b46-531383699898 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:00.658897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da714b0b-c511-4ef5-8b46-531383699898 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:00.659320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da714b0b-c511-4ef5-8b46-531383699898 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:00.660757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da714b0b-c511-4ef5-8b46-531383699898 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:00.677700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 300/589] 10.222.0.121 () {56 vars in 1144 bytes} [Thu May 26 17:40:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:00.678494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da714b0b-c511-4ef5-8b46-531383699898 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:00.718045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41398893-c03f-47a8-8ce4-5c48f4983ce8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:00.740346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:00.754488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a108ab9-2a5a-472f-9e07-e43dc74b473d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:00.755115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a108ab9-2a5a-472f-9e07-e43dc74b473d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:00.755398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a108ab9-2a5a-472f-9e07-e43dc74b473d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:00.755679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a108ab9-2a5a-472f-9e07-e43dc74b473d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:00.756513 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a108ab9-2a5a-472f-9e07-e43dc74b473d None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=eb9ba9c73956403ba870f6f6b0e687aa) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:00.756825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a108ab9-2a5a-472f-9e07-e43dc74b473d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:00.758479 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a108ab9-2a5a-472f-9e07-e43dc74b473d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:00.764203 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 290/590] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:00 2022] GET /identity/v3/role_assignments?user.id=eb9ba9c73956403ba870f6f6b0e687aa&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:00.764982 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a108ab9-2a5a-472f-9e07-e43dc74b473d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:00.792790 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da714b0b-c511-4ef5-8b46-531383699898 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:00.816909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:00.835740 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89edaedf-65bb-49e7-8c86-f6cab40389d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:00.837259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edaedf-65bb-49e7-8c86-f6cab40389d7 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:00.837964 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edaedf-65bb-49e7-8c86-f6cab40389d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:00.838410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edaedf-65bb-49e7-8c86-f6cab40389d7 None admin] PATH_INFO: `/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/683c54f489f440b9842cb94980214df1/roles/9399fac5b71b46f79d1ebe5142fc1176` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:00.857386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89edaedf-65bb-49e7-8c86-f6cab40389d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61dea65b34b47a186151c6a63bc7d79, user_id=683c54f489f440b9842cb94980214df1, role_id=9399fac5b71b46f79d1ebe5142fc1176)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:00.859296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89edaedf-65bb-49e7-8c86-f6cab40389d7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:00.888014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a108ab9-2a5a-472f-9e07-e43dc74b473d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:00.926293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68217c-9c5f-4f97-8b8d-6229cb85e92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:00.927225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68217c-9c5f-4f97-8b8d-6229cb85e92b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:00.927503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68217c-9c5f-4f97-8b8d-6229cb85e92b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:00.927775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68217c-9c5f-4f97-8b8d-6229cb85e92b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:00.935574 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68217c-9c5f-4f97-8b8d-6229cb85e92b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:00.937055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68217c-9c5f-4f97-8b8d-6229cb85e92b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:00.942548 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 301/591] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:40:00 2022] PUT /identity/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/683c54f489f440b9842cb94980214df1/roles/9399fac5b71b46f79d1ebe5142fc1176 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:40:00.943909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89edaedf-65bb-49e7-8c86-f6cab40389d7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-8e68217c-9c5f-4f97-8b8d-6229cb85e92b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:00.957493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:00.958519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:00.959979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:00.959979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:00.959979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:00.959979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:00.959979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:00.959979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:00.959979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:00.959979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:00.959979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 291/592] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:00.959979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68217c-9c5f-4f97-8b8d-6229cb85e92b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:01.088068 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:01.092932 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68217c-9c5f-4f97-8b8d-6229cb85e92b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:01.101706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89edaedf-65bb-49e7-8c86-f6cab40389d7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:01.121526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:01.139210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cdc4d-e9a8-442c-8304-093d7b1c1f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:01.140826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cdc4d-e9a8-442c-8304-093d7b1c1f04 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:01.141490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cdc4d-e9a8-442c-8304-093d7b1c1f04 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:01.142270 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cdc4d-e9a8-442c-8304-093d7b1c1f04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:01.153080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c9bc9-389f-480e-b0f2-2a0e5d684995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:01.153911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c9bc9-389f-480e-b0f2-2a0e5d684995 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:01.154295 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c9bc9-389f-480e-b0f2-2a0e5d684995 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:01.154674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c9bc9-389f-480e-b0f2-2a0e5d684995 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:01.156043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c9bc9-389f-480e-b0f2-2a0e5d684995 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:01.156713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c9bc9-389f-480e-b0f2-2a0e5d684995 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:01.157793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9cdc4d-e9a8-442c-8304-093d7b1c1f04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:01.158346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c9bc9-389f-480e-b0f2-2a0e5d684995 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:01.159920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9cdc4d-e9a8-442c-8304-093d7b1c1f04 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:01.173259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 302/593] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:01.174733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c9bc9-389f-480e-b0f2-2a0e5d684995 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-4d9cdc4d-e9a8-442c-8304-093d7b1c1f04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:01.188222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:01.189352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:01.191358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:01.191358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:01.191358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:01.191358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:01.192005 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 292/594] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:01.193026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cdc4d-e9a8-442c-8304-093d7b1c1f04 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:01.289147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c9bc9-389f-480e-b0f2-2a0e5d684995 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:01.320675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cdc4d-e9a8-442c-8304-093d7b1c1f04 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:01.327523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a70989-4432-4068-ad20-2e3d49819c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:01.328204 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a70989-4432-4068-ad20-2e3d49819c18 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:01.328485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a70989-4432-4068-ad20-2e3d49819c18 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:01.328762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a70989-4432-4068-ad20-2e3d49819c18 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:01.330530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a70989-4432-4068-ad20-2e3d49819c18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:01.330888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a70989-4432-4068-ad20-2e3d49819c18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:01.333973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a70989-4432-4068-ad20-2e3d49819c18 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:01.352145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 303/595] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:01.353419 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a70989-4432-4068-ad20-2e3d49819c18 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:01.368778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d91a8f-beb4-4ef2-b50a-0e50e72795cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:01.369810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d91a8f-beb4-4ef2-b50a-0e50e72795cd None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:01.370187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d91a8f-beb4-4ef2-b50a-0e50e72795cd None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:01.370561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d91a8f-beb4-4ef2-b50a-0e50e72795cd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:01.379832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d91a8f-beb4-4ef2-b50a-0e50e72795cd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:01.381583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d91a8f-beb4-4ef2-b50a-0e50e72795cd None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:01.393687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-c3d91a8f-beb4-4ef2-b50a-0e50e72795cd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:01.402439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:01.409868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:01.410808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:01.410808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:01.410808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:01.410808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:01.410808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:01.410808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:01.410808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:01.410808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:01.410808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:40:01.410808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:01.411371 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 293/596] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:40:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:01.411900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d91a8f-beb4-4ef2-b50a-0e50e72795cd None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:01.428547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:01.487339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a70989-4432-4068-ad20-2e3d49819c18 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:01.538517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c004223-d109-4d15-afd0-c93a1793dba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:01.539549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c004223-d109-4d15-afd0-c93a1793dba0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:01.540007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c004223-d109-4d15-afd0-c93a1793dba0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:01.540428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d91a8f-beb4-4ef2-b50a-0e50e72795cd None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:01.540666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c004223-d109-4d15-afd0-c93a1793dba0 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:01.541807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c004223-d109-4d15-afd0-c93a1793dba0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:01.542374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c004223-d109-4d15-afd0-c93a1793dba0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:01.544041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c004223-d109-4d15-afd0-c93a1793dba0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:01.558156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 304/597] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:01.559361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c004223-d109-4d15-afd0-c93a1793dba0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:01.583859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219122b3-8d0f-4be8-adc4-87bd8243e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:01.584874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219122b3-8d0f-4be8-adc4-87bd8243e9f2 None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:01.585227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219122b3-8d0f-4be8-adc4-87bd8243e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:01.585663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219122b3-8d0f-4be8-adc4-87bd8243e9f2 None admin] PATH_INFO: `/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/c0889a69a018452d8de1924072739d17/roles/9399fac5b71b46f79d1ebe5142fc1176` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:01.604099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219122b3-8d0f-4be8-adc4-87bd8243e9f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61dea65b34b47a186151c6a63bc7d79, user_id=c0889a69a018452d8de1924072739d17, role_id=9399fac5b71b46f79d1ebe5142fc1176)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:01.605715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219122b3-8d0f-4be8-adc4-87bd8243e9f2 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:01.659501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:01.668549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c004223-d109-4d15-afd0-c93a1793dba0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:01.694836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 294/598] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:40:01 2022] PUT /identity/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/c0889a69a018452d8de1924072739d17/roles/9399fac5b71b46f79d1ebe5142fc1176 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:40:01.696236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219122b3-8d0f-4be8-adc4-87bd8243e9f2 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:01.722981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d52bc-300f-4cdf-86d9-96820755e955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:01.723910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d52bc-300f-4cdf-86d9-96820755e955 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:01.725614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d52bc-300f-4cdf-86d9-96820755e955 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:01.725962 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d52bc-300f-4cdf-86d9-96820755e955 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:01.728125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d52bc-300f-4cdf-86d9-96820755e955 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:01.729236 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d52bc-300f-4cdf-86d9-96820755e955 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:01.734218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d52bc-300f-4cdf-86d9-96820755e955 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:01.763903 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 305/599] 10.222.0.121 () {56 vars in 1142 bytes} [Thu May 26 17:40:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:01.769724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d52bc-300f-4cdf-86d9-96820755e955 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:01.849392 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:01.911402 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219122b3-8d0f-4be8-adc4-87bd8243e9f2 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:01.929842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d52bc-300f-4cdf-86d9-96820755e955 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:01.950401 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286019e-861b-4b81-8600-612996a85448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:01.951241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286019e-861b-4b81-8600-612996a85448 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:01.951522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286019e-861b-4b81-8600-612996a85448 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:01.951903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286019e-861b-4b81-8600-612996a85448 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:01.962117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286019e-861b-4b81-8600-612996a85448 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:01.963613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286019e-861b-4b81-8600-612996a85448 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:01.975951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dceb220-585c-4bc1-8899-179eba6f520a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:01.976910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceb220-585c-4bc1-8899-179eba6f520a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:01.977292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceb220-585c-4bc1-8899-179eba6f520a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:01.977671 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceb220-585c-4bc1-8899-179eba6f520a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-6286019e-861b-4b81-8600-612996a85448 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:01.979892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:01.980969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:01.982059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:01.982059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:01.982059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:01.982059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:01.982059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:01.982059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:01.982059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:01.982059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:01.982059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 295/600] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:01.988978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286019e-861b-4b81-8600-612996a85448 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:01.992444 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dceb220-585c-4bc1-8899-179eba6f520a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:01.994234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dceb220-585c-4bc1-8899-179eba6f520a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-9dceb220-585c-4bc1-8899-179eba6f520a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:02.016044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:02.017167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:02.018330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:02.018330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:02.018330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:02.018330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:02.019326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 306/601] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:02.025829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dceb220-585c-4bc1-8899-179eba6f520a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:02.028681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:02.131049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286019e-861b-4b81-8600-612996a85448 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:02.133324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dceb220-585c-4bc1-8899-179eba6f520a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:02.169667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a9bce-3403-454c-9dcb-da22258acf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:02.170259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a9bce-3403-454c-9dcb-da22258acf19 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:02.170538 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a9bce-3403-454c-9dcb-da22258acf19 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:02.170810 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a9bce-3403-454c-9dcb-da22258acf19 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:02.171558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a9bce-3403-454c-9dcb-da22258acf19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:02.171861 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a9bce-3403-454c-9dcb-da22258acf19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:02.173233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a9bce-3403-454c-9dcb-da22258acf19 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:02.174573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:02.186696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 307/602] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:02.192869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a9bce-3403-454c-9dcb-da22258acf19 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:02.205332 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a965383-fd9f-4cc5-a705-b6b94f9d11ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:02.207210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a965383-fd9f-4cc5-a705-b6b94f9d11ca None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:02.207701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a965383-fd9f-4cc5-a705-b6b94f9d11ca None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:02.208229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a965383-fd9f-4cc5-a705-b6b94f9d11ca None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:02.209341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a965383-fd9f-4cc5-a705-b6b94f9d11ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:02.209983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a965383-fd9f-4cc5-a705-b6b94f9d11ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:02.211712 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a965383-fd9f-4cc5-a705-b6b94f9d11ca None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:02.237220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 296/603] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:02.243092 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a965383-fd9f-4cc5-a705-b6b94f9d11ca None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:02.275328 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:02.357904 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a9bce-3403-454c-9dcb-da22258acf19 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:02.391173 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a965383-fd9f-4cc5-a705-b6b94f9d11ca None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:02.437405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160c372-8a14-4279-854f-6094b20c7aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:02.439008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160c372-8a14-4279-854f-6094b20c7aa6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:02.441430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160c372-8a14-4279-854f-6094b20c7aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:02.441960 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160c372-8a14-4279-854f-6094b20c7aa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:02.446061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b78d0-e0c9-4d16-8b35-5ad1db3d6cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:02.448612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b78d0-e0c9-4d16-8b35-5ad1db3d6cfd None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:02.451453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b78d0-e0c9-4d16-8b35-5ad1db3d6cfd None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:02.451955 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b78d0-e0c9-4d16-8b35-5ad1db3d6cfd None admin] PATH_INFO: `/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/08709b462300437598c8db331e5ad20f/roles/9399fac5b71b46f79d1ebe5142fc1176` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:02.458319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160c372-8a14-4279-854f-6094b20c7aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:02.464825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160c372-8a14-4279-854f-6094b20c7aa6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:02.485658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b78d0-e0c9-4d16-8b35-5ad1db3d6cfd None admin] RBAC: Authorizing `identity:create_grant(project_id=d61dea65b34b47a186151c6a63bc7d79, user_id=08709b462300437598c8db331e5ad20f, role_id=9399fac5b71b46f79d1ebe5142fc1176)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:02.488577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b78d0-e0c9-4d16-8b35-5ad1db3d6cfd None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-9160c372-8a14-4279-854f-6094b20c7aa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:02.491304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:02.492179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:02.493062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:02.493062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:02.493062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:02.493062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:02.493062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:02.493062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:02.493062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:02.493062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:02.496013 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 308/604] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:02.501549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160c372-8a14-4279-854f-6094b20c7aa6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:02.618580 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 297/605] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:40:02 2022] PUT /identity/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/08709b462300437598c8db331e5ad20f/roles/9399fac5b71b46f79d1ebe5142fc1176 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:40:02.633100 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160c372-8a14-4279-854f-6094b20c7aa6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:02.669686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:02.690575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce528c-dac6-4037-a988-44ec4a16a930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:02.691765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce528c-dac6-4037-a988-44ec4a16a930 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:02.692272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce528c-dac6-4037-a988-44ec4a16a930 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:02.692751 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce528c-dac6-4037-a988-44ec4a16a930 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:02.693768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce528c-dac6-4037-a988-44ec4a16a930 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:02.694350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce528c-dac6-4037-a988-44ec4a16a930 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:02.696144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce528c-dac6-4037-a988-44ec4a16a930 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:02.715586 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 309/606] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:02.722041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b78d0-e0c9-4d16-8b35-5ad1db3d6cfd None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:02.791538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:02.922991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b78d0-e0c9-4d16-8b35-5ad1db3d6cfd None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:02.970282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34612bd-2184-46e5-88e8-f55dcce3a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:02.972333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34612bd-2184-46e5-88e8-f55dcce3a322 None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:02.972824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34612bd-2184-46e5-88e8-f55dcce3a322 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:02.973300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34612bd-2184-46e5-88e8-f55dcce3a322 None admin] PATH_INFO: `/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/25d85d02abed40199dd2515ecf84b7fb/roles/9399fac5b71b46f79d1ebe5142fc1176` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:02.996133 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34612bd-2184-46e5-88e8-f55dcce3a322 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61dea65b34b47a186151c6a63bc7d79, user_id=25d85d02abed40199dd2515ecf84b7fb, role_id=9399fac5b71b46f79d1ebe5142fc1176)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:02.998594 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34612bd-2184-46e5-88e8-f55dcce3a322 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:03.102850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 298/607] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:40:02 2022] PUT /identity/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/25d85d02abed40199dd2515ecf84b7fb/roles/9399fac5b71b46f79d1ebe5142fc1176 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:40:03.461160 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7ac61-64f4-4528-9b53-15751426c30c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:03.461698 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7ac61-64f4-4528-9b53-15751426c30c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:03.461698 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7ac61-64f4-4528-9b53-15751426c30c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:03.463449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 310/608] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:03.469486 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b3eb-f7ff-4af5-9ea9-0660e47482db None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:03.469969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b3eb-f7ff-4af5-9ea9-0660e47482db None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:03.470046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b3eb-f7ff-4af5-9ea9-0660e47482db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:03.490301 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-4ad1b3eb-f7ff-4af5-9ea9-0660e47482db None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:03.491564 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4ad1b3eb-f7ff-4af5-9ea9-0660e47482db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:03.576466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 299/609] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:40:03.599851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7ac61-64f4-4528-9b53-15751426c30c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:03.734682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bd22c-7783-4140-a4ea-6a281dcb6b27 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:03.735010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bd22c-7783-4140-a4ea-6a281dcb6b27 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:03.735202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bd22c-7783-4140-a4ea-6a281dcb6b27 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:03.736925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 300/610] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:03.742669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7ac61-64f4-4528-9b53-15751426c30c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:03.742869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbb696-11cb-44bf-8068-8cc955b40819 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:03.743094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbb696-11cb-44bf-8068-8cc955b40819 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:03.744458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbb696-11cb-44bf-8068-8cc955b40819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:03.779380 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-7fbbb696-11cb-44bf-8068-8cc955b40819 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:03.780569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7fbbb696-11cb-44bf-8068-8cc955b40819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:03.784963 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:03.786484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e9da3-75d4-4d74-a761-e964f8f17fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:03.787615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e9da3-75d4-4d74-a761-e964f8f17fc7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:03.788105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e9da3-75d4-4d74-a761-e964f8f17fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:03.788582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e9da3-75d4-4d74-a761-e964f8f17fc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:03.816129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e9da3-75d4-4d74-a761-e964f8f17fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-d06e9da3-75d4-4d74-a761-e964f8f17fc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:03.831296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:03.832311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:03.833161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 311/611] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:40:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:03.839247 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e9da3-75d4-4d74-a761-e964f8f17fc7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:03.925225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 301/612] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:40:03.926445 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:03.972014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbbb696-11cb-44bf-8068-8cc955b40819 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:03.989651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e9da3-75d4-4d74-a761-e964f8f17fc7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:04.039585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5229884-a935-469b-a693-96ec2563d18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:04.040572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5229884-a935-469b-a693-96ec2563d18d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:04.040572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5229884-a935-469b-a693-96ec2563d18d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:04.041593 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5229884-a935-469b-a693-96ec2563d18d None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:04.042310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5229884-a935-469b-a693-96ec2563d18d None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:04.042614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5229884-a935-469b-a693-96ec2563d18d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:04.044199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5229884-a935-469b-a693-96ec2563d18d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:04.061445 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 312/613] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:40:03 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:04.067344 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5229884-a935-469b-a693-96ec2563d18d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:04.123621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbbb696-11cb-44bf-8068-8cc955b40819 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:04.171593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6df78-8fbb-4d8a-807f-8006f929057a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:04.172415 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6df78-8fbb-4d8a-807f-8006f929057a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:04.172692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6df78-8fbb-4d8a-807f-8006f929057a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:04.172963 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6df78-8fbb-4d8a-807f-8006f929057a None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:04.212243 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:04.223522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6df78-8fbb-4d8a-807f-8006f929057a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d8f6df78-8fbb-4d8a-807f-8006f929057a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:04.231407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:04.232603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:04.233477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 302/614] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:40:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:04.239102 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6df78-8fbb-4d8a-807f-8006f929057a None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:04.250562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5229884-a935-469b-a693-96ec2563d18d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:04.255034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:04.303803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e13201-c02e-4c22-bb78-30381615fb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:04.304642 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e13201-c02e-4c22-bb78-30381615fb70 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:04.304927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e13201-c02e-4c22-bb78-30381615fb70 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:04.305205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e13201-c02e-4c22-bb78-30381615fb70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:04.332603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e13201-c02e-4c22-bb78-30381615fb70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:04.334170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e13201-c02e-4c22-bb78-30381615fb70 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-26e13201-c02e-4c22-bb78-30381615fb70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:04.369430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:04.370417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:04.371347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:04.371347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:04.371347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:04.371347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:04.379389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 313/615] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:04.384558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e13201-c02e-4c22-bb78-30381615fb70 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:04.516726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e13201-c02e-4c22-bb78-30381615fb70 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:04.560242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87363afb-5505-4d7e-87ac-b2ba4589bc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:04.560933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87363afb-5505-4d7e-87ac-b2ba4589bc29 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:04.561214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87363afb-5505-4d7e-87ac-b2ba4589bc29 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:04.561489 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87363afb-5505-4d7e-87ac-b2ba4589bc29 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:04.562221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87363afb-5505-4d7e-87ac-b2ba4589bc29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:04.562527 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87363afb-5505-4d7e-87ac-b2ba4589bc29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:04.563947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87363afb-5505-4d7e-87ac-b2ba4589bc29 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:04.591373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 314/616] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:04.592718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec5d26-1feb-4d82-a427-ba606d95f9ea None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:04.593001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec5d26-1feb-4d82-a427-ba606d95f9ea None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:04.593274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec5d26-1feb-4d82-a427-ba606d95f9ea None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:04.599552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 315/617] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:04 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:04.600378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec5d26-1feb-4d82-a427-ba606d95f9ea None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:04.602330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6df78-8fbb-4d8a-807f-8006f929057a None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:04.623128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:04.646852 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53edd78f-3539-4245-9b8d-cbd5cc0b25da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:04.654160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53edd78f-3539-4245-9b8d-cbd5cc0b25da None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:04.654160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53edd78f-3539-4245-9b8d-cbd5cc0b25da None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:04.654160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53edd78f-3539-4245-9b8d-cbd5cc0b25da None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:04.654160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53edd78f-3539-4245-9b8d-cbd5cc0b25da None admin] RBAC: Adding query filter params (name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:04.654160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53edd78f-3539-4245-9b8d-cbd5cc0b25da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:04.654160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53edd78f-3539-4245-9b8d-cbd5cc0b25da None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:04.664366 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 303/618] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:40:04 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:04.666791 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074f97c-f3dd-445c-a243-5ef591260da1 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:04.667229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074f97c-f3dd-445c-a243-5ef591260da1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:04.667783 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074f97c-f3dd-445c-a243-5ef591260da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:04.687114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5074f97c-f3dd-445c-a243-5ef591260da1 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:04.688188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5074f97c-f3dd-445c-a243-5ef591260da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:04.724142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec5d26-1feb-4d82-a427-ba606d95f9ea None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:04.747898 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:04.762385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 304/619] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:40:04.763426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbad6e8-6584-45cb-9cef-ee1fdd0afc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:04.764462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbad6e8-6584-45cb-9cef-ee1fdd0afc85 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:04.764902 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbad6e8-6584-45cb-9cef-ee1fdd0afc85 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:04.765331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbad6e8-6584-45cb-9cef-ee1fdd0afc85 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:04.766659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074f97c-f3dd-445c-a243-5ef591260da1 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:04.775205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbad6e8-6584-45cb-9cef-ee1fdd0afc85 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:04.777176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbad6e8-6584-45cb-9cef-ee1fdd0afc85 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-2dbad6e8-6584-45cb-9cef-ee1fdd0afc85 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:04.793875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:04.796170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:04.797144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:04.797144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:04.797144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:04.797144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:04.797144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:04.797144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:04.797144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:04.797144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:04.797144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 17:40:04.797144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:04.798618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 316/620] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:40:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:04.805530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbad6e8-6584-45cb-9cef-ee1fdd0afc85 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:04.891190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074f97c-f3dd-445c-a243-5ef591260da1 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:04.903248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:04.918260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbad6e8-6584-45cb-9cef-ee1fdd0afc85 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:04.928955 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4c3a2-c9a8-41f8-967a-4be0cebb6775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:04.929761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4c3a2-c9a8-41f8-967a-4be0cebb6775 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:04.930038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4c3a2-c9a8-41f8-967a-4be0cebb6775 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:04.930308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4c3a2-c9a8-41f8-967a-4be0cebb6775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:04.941984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4c3a2-c9a8-41f8-967a-4be0cebb6775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:04.943465 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4c3a2-c9a8-41f8-967a-4be0cebb6775 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:04.959241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad082f5-8f8a-4829-b28a-ff1a658b84e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:04.960278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad082f5-8f8a-4829-b28a-ff1a658b84e7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:04.960736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad082f5-8f8a-4829-b28a-ff1a658b84e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:04.961210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad082f5-8f8a-4829-b28a-ff1a658b84e7 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-dee4c3a2-c9a8-41f8-967a-4be0cebb6775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:04.961520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:04.962374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:04.963430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:04.963430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:04.963430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:04.963430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:04.963430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 305/621] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:04.963937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4c3a2-c9a8-41f8-967a-4be0cebb6775 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:04.964798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad082f5-8f8a-4829-b28a-ff1a658b84e7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:04.965256 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad082f5-8f8a-4829-b28a-ff1a658b84e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:04.967763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad082f5-8f8a-4829-b28a-ff1a658b84e7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:04.988506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 317/622] 10.222.0.121 () {56 vars in 1140 bytes} [Thu May 26 17:40:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:04.989939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad082f5-8f8a-4829-b28a-ff1a658b84e7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:05.019211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:05.097689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4c3a2-c9a8-41f8-967a-4be0cebb6775 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:05.116530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad082f5-8f8a-4829-b28a-ff1a658b84e7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:05.145420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db1075-0f6e-4a87-a875-635b8005507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:05.146425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db1075-0f6e-4a87-a875-635b8005507a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:05.146740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db1075-0f6e-4a87-a875-635b8005507a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:05.147133 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db1075-0f6e-4a87-a875-635b8005507a None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:05.159646 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db1075-0f6e-4a87-a875-635b8005507a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:05.161709 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a1398-735b-4a29-95f5-3596195da1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:05.162545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1398-735b-4a29-95f5-3596195da1b9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:05.163018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1398-735b-4a29-95f5-3596195da1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:05.163452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1398-735b-4a29-95f5-3596195da1b9 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:05.164329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a1398-735b-4a29-95f5-3596195da1b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:05.164772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a1398-735b-4a29-95f5-3596195da1b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:05.166542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a1398-735b-4a29-95f5-3596195da1b9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:05.168437 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-53db1075-0f6e-4a87-a875-635b8005507a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:05.168745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:05.169790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:05.170793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:05.170793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:05.170793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:05.170793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:05.170793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:05.170793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:05.170793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:05.170793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:05.170793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:05.170793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:05.170793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:05.170793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:05.174232 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 306/623] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:40:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:05.174232 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db1075-0f6e-4a87-a875-635b8005507a None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:05.187039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 318/624] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:05.188045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a1398-735b-4a29-95f5-3596195da1b9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:05.295583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db1075-0f6e-4a87-a875-635b8005507a None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:05.297178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a1398-735b-4a29-95f5-3596195da1b9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:05.298594 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:05.338865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7227ad-fa79-4fff-9ac6-91acfdde2099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:05.339144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a40a3-744c-45fd-bd05-0a3f4db1f4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:05.339928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a40a3-744c-45fd-bd05-0a3f4db1f4cc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:05.340248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7227ad-fa79-4fff-9ac6-91acfdde2099 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:05.340456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a40a3-744c-45fd-bd05-0a3f4db1f4cc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:05.340814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7227ad-fa79-4fff-9ac6-91acfdde2099 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:05.340924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a40a3-744c-45fd-bd05-0a3f4db1f4cc None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:05.341502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7227ad-fa79-4fff-9ac6-91acfdde2099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:05.341877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a40a3-744c-45fd-bd05-0a3f4db1f4cc None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:05.342297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a40a3-744c-45fd-bd05-0a3f4db1f4cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:05.343780 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a40a3-744c-45fd-bd05-0a3f4db1f4cc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:05.358734 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 319/625] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:40:05 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:05.358734 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a40a3-744c-45fd-bd05-0a3f4db1f4cc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:05.359321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7227ad-fa79-4fff-9ac6-91acfdde2099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:05.361027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7227ad-fa79-4fff-9ac6-91acfdde2099 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-8c7227ad-fa79-4fff-9ac6-91acfdde2099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:05.381075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:05.382022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:05.382927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:05.382927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:05.382927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:05.382927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:05.385105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 307/626] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:05.386171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7227ad-fa79-4fff-9ac6-91acfdde2099 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:05.430647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:05.481305 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a40a3-744c-45fd-bd05-0a3f4db1f4cc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:05.511424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7227ad-fa79-4fff-9ac6-91acfdde2099 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:05.519266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76327db8-b39c-4bee-b0f8-4e7113719756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:05.520077 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76327db8-b39c-4bee-b0f8-4e7113719756 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:05.520356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76327db8-b39c-4bee-b0f8-4e7113719756 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:05.520627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76327db8-b39c-4bee-b0f8-4e7113719756 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:05.529124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76327db8-b39c-4bee-b0f8-4e7113719756 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:05.530598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76327db8-b39c-4bee-b0f8-4e7113719756 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-76327db8-b39c-4bee-b0f8-4e7113719756 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:05.551721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:05.553946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:05.555082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:05.555082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:05.555082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:05.555082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:05.555082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:05.555082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:05.555082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:05.555082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:05.555082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 17:40:05.555082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:05.557557 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 320/627] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:40:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:05.558682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76327db8-b39c-4bee-b0f8-4e7113719756 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:05.562347 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d381e33-2a5f-401a-9dde-d3c4640f72b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:05.563183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d381e33-2a5f-401a-9dde-d3c4640f72b8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:05.563466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d381e33-2a5f-401a-9dde-d3c4640f72b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:05.563752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d381e33-2a5f-401a-9dde-d3c4640f72b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:05.577741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d381e33-2a5f-401a-9dde-d3c4640f72b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:05.579214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d381e33-2a5f-401a-9dde-d3c4640f72b8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-9d381e33-2a5f-401a-9dde-d3c4640f72b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:05.601543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:05.602935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:05.603836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:05.603836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:05.603836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:05.603836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:05.603836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 308/628] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:05.604524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d92b6-966a-4a6f-8c6e-de02713d1779 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:05.604620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d92b6-966a-4a6f-8c6e-de02713d1779 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:05.604861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d92b6-966a-4a6f-8c6e-de02713d1779 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:05.610168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 309/629] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:05 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:05.610903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d92b6-966a-4a6f-8c6e-de02713d1779 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:05.622223 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:05.685044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76327db8-b39c-4bee-b0f8-4e7113719756 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:05.707639 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:05.727540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4065c-dde1-478d-9bf6-24f1a25da0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:05.728247 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4065c-dde1-478d-9bf6-24f1a25da0df None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:05.728524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4065c-dde1-478d-9bf6-24f1a25da0df None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:05.728787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4065c-dde1-478d-9bf6-24f1a25da0df None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:05.729508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4065c-dde1-478d-9bf6-24f1a25da0df None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:05.729812 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4065c-dde1-478d-9bf6-24f1a25da0df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:05.731237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4065c-dde1-478d-9bf6-24f1a25da0df None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:05.741504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d92b6-966a-4a6f-8c6e-de02713d1779 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:05.746429 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 321/630] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:05.747296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4065c-dde1-478d-9bf6-24f1a25da0df None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:05.781154 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf9f51-f8d4-47b1-8b04-11bc282a1b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:05.782163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf9f51-f8d4-47b1-8b04-11bc282a1b14 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:05.782656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf9f51-f8d4-47b1-8b04-11bc282a1b14 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:05.783231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf9f51-f8d4-47b1-8b04-11bc282a1b14 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:05.784506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf9f51-f8d4-47b1-8b04-11bc282a1b14 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:05.785019 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf9f51-f8d4-47b1-8b04-11bc282a1b14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:05.786449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf9f51-f8d4-47b1-8b04-11bc282a1b14 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:05.812399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 310/631] 10.222.0.121 () {56 vars in 1142 bytes} [Thu May 26 17:40:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:05.812399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bb2dd-e3cb-42e8-adf5-1dd4cee9a0af None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:05.812399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bb2dd-e3cb-42e8-adf5-1dd4cee9a0af None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:05.812399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bb2dd-e3cb-42e8-adf5-1dd4cee9a0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:05.827450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-921bb2dd-e3cb-42e8-adf5-1dd4cee9a0af None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:05.828626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-921bb2dd-e3cb-42e8-adf5-1dd4cee9a0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:05.870551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4065c-dde1-478d-9bf6-24f1a25da0df None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:05.893965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:05.906742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ed6c3-6dff-4ada-8a5c-cca5d9559e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:05.907437 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ed6c3-6dff-4ada-8a5c-cca5d9559e5e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:05.907534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 311/632] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:40:05.907728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ed6c3-6dff-4ada-8a5c-cca5d9559e5e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:05.908841 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ed6c3-6dff-4ada-8a5c-cca5d9559e5e None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:05.908952 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bb2dd-e3cb-42e8-adf5-1dd4cee9a0af None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:05.910024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ed6c3-6dff-4ada-8a5c-cca5d9559e5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:05.910337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ed6c3-6dff-4ada-8a5c-cca5d9559e5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:05.911759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ed6c3-6dff-4ada-8a5c-cca5d9559e5e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:05.925594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 322/633] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:05.931330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ed6c3-6dff-4ada-8a5c-cca5d9559e5e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:05.967431 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:06.016259 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bb2dd-e3cb-42e8-adf5-1dd4cee9a0af None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:06.033999 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ed6c3-6dff-4ada-8a5c-cca5d9559e5e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:06.051089 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcf965-39b5-45eb-8dc8-2ca3ea9e043d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:06.051971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcf965-39b5-45eb-8dc8-2ca3ea9e043d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:06.052251 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcf965-39b5-45eb-8dc8-2ca3ea9e043d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:06.052529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcf965-39b5-45eb-8dc8-2ca3ea9e043d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:06.060231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcf965-39b5-45eb-8dc8-2ca3ea9e043d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:06.061656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcf965-39b5-45eb-8dc8-2ca3ea9e043d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:06.067838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd349ee-2d10-47ff-aba6-8b1a9098b9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:06.068483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd349ee-2d10-47ff-aba6-8b1a9098b9a8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:06.068762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd349ee-2d10-47ff-aba6-8b1a9098b9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:06.069031 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd349ee-2d10-47ff-aba6-8b1a9098b9a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-6efcf965-39b5-45eb-8dc8-2ca3ea9e043d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:06.076091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:06.076985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:06.077899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:06.077899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:06.077899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:06.077899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:06.077899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:06.077899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:06.077899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:06.077899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:06.077899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 312/634] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:06.078896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5eacb-b86a-42c3-ba41-fccee9751bd5 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:06.079111 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5eacb-b86a-42c3-ba41-fccee9751bd5 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:06.079386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5eacb-b86a-42c3-ba41-fccee9751bd5 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:06.081253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 313/635] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:06.081946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5eacb-b86a-42c3-ba41-fccee9751bd5 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:06.144616 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:06.148469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd349ee-2d10-47ff-aba6-8b1a9098b9a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:06.149911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd349ee-2d10-47ff-aba6-8b1a9098b9a8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:06.184958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:06.189065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5eacb-b86a-42c3-ba41-fccee9751bd5 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:06.226286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cbdf9-415b-416c-9eeb-3ed6516bacdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:06.227118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cbdf9-415b-416c-9eeb-3ed6516bacdf None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:06.227400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cbdf9-415b-416c-9eeb-3ed6516bacdf None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:06.227673 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cbdf9-415b-416c-9eeb-3ed6516bacdf None admin] PATH_INFO: `/v3/users/nova` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:06.235484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48cbdf9-415b-416c-9eeb-3ed6516bacdf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:06.237519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48cbdf9-415b-416c-9eeb-3ed6516bacdf None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-f48cbdf9-415b-416c-9eeb-3ed6516bacdf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:06.253220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:06.254298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:06.255278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:06.255278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:06.255278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:06.255278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:06.255278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:06.255278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:06.255278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:06.255278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:06.255278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:40:06.255278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:06.255278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 314/636] 10.222.0.121 () {56 vars in 1098 bytes} [Thu May 26 17:40:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:06.255278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cbdf9-415b-416c-9eeb-3ed6516bacdf None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:06.264793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 323/637] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:05 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:40:06.264793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd349ee-2d10-47ff-aba6-8b1a9098b9a8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:06.362969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cbdf9-415b-416c-9eeb-3ed6516bacdf None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:06.370266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd349ee-2d10-47ff-aba6-8b1a9098b9a8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:06.382521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:06.403190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ecf8d-9054-42b7-9604-26d235d1fd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:06.403961 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ecf8d-9054-42b7-9604-26d235d1fd78 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:06.404146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ecf8d-9054-42b7-9604-26d235d1fd78 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:06.404414 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ecf8d-9054-42b7-9604-26d235d1fd78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:06.407425 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd407d-a0a7-4bde-a27c-3e44606de087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:06.408764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd407d-a0a7-4bde-a27c-3e44606de087 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:06.409045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd407d-a0a7-4bde-a27c-3e44606de087 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:06.409336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd407d-a0a7-4bde-a27c-3e44606de087 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:06.410186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd407d-a0a7-4bde-a27c-3e44606de087 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:06.410492 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd407d-a0a7-4bde-a27c-3e44606de087 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:06.411976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd407d-a0a7-4bde-a27c-3e44606de087 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:06.414550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:06.416427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ecf8d-9054-42b7-9604-26d235d1fd78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-2f0ecf8d-9054-42b7-9604-26d235d1fd78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:06.424114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:06.430474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 315/638] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:40:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:06.431364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439332cd-68cc-4b3a-ab91-b624bd907f0c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:06.432127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439332cd-68cc-4b3a-ab91-b624bd907f0c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:06.432127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439332cd-68cc-4b3a-ab91-b624bd907f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:06.432260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 324/639] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:06.432260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd407d-a0a7-4bde-a27c-3e44606de087 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:06.445105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-439332cd-68cc-4b3a-ab91-b624bd907f0c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:06.446296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-439332cd-68cc-4b3a-ab91-b624bd907f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:06.515548 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 316/640] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:40:06.520232 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439332cd-68cc-4b3a-ab91-b624bd907f0c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:06.531366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd407d-a0a7-4bde-a27c-3e44606de087 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:06.568623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd53c52-755e-46ea-9fd8-8e19330ef599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:06.569330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd53c52-755e-46ea-9fd8-8e19330ef599 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:06.569614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd53c52-755e-46ea-9fd8-8e19330ef599 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:06.569886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd53c52-755e-46ea-9fd8-8e19330ef599 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:06.570720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd53c52-755e-46ea-9fd8-8e19330ef599 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:06.571051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd53c52-755e-46ea-9fd8-8e19330ef599 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:06.572487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd53c52-755e-46ea-9fd8-8e19330ef599 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:06.591184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 325/641] 10.222.0.121 () {56 vars in 1138 bytes} [Thu May 26 17:40:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:06.591989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd53c52-755e-46ea-9fd8-8e19330ef599 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:06.632410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439332cd-68cc-4b3a-ab91-b624bd907f0c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:06.639916 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:06.652661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:06.669449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e64ce7-bee7-48f0-aa3b-67c02b75832b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:06.670170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e64ce7-bee7-48f0-aa3b-67c02b75832b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:06.670451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e64ce7-bee7-48f0-aa3b-67c02b75832b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:06.670720 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e64ce7-bee7-48f0-aa3b-67c02b75832b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:06.681983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e64ce7-bee7-48f0-aa3b-67c02b75832b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:06.683467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e64ce7-bee7-48f0-aa3b-67c02b75832b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d8e64ce7-bee7-48f0-aa3b-67c02b75832b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:06.701427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:06.702598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:06.703520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:06.703520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:06.703520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:06.703520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:06.703755 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd53c52-755e-46ea-9fd8-8e19330ef599 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:06.703837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 317/642] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:06.703837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e64ce7-bee7-48f0-aa3b-67c02b75832b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:06.741184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28864ff-a175-4f6c-bcef-0b4ad9bc2712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:06.742062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28864ff-a175-4f6c-bcef-0b4ad9bc2712 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:06.742364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28864ff-a175-4f6c-bcef-0b4ad9bc2712 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:06.744561 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28864ff-a175-4f6c-bcef-0b4ad9bc2712 None admin] PATH_INFO: `/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/eb9ba9c73956403ba870f6f6b0e687aa/roles/9399fac5b71b46f79d1ebe5142fc1176` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:06.758618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28864ff-a175-4f6c-bcef-0b4ad9bc2712 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61dea65b34b47a186151c6a63bc7d79, user_id=eb9ba9c73956403ba870f6f6b0e687aa, role_id=9399fac5b71b46f79d1ebe5142fc1176)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:06.760103 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28864ff-a175-4f6c-bcef-0b4ad9bc2712 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:06.812844 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 326/643] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:40:06 2022] PUT /identity/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/eb9ba9c73956403ba870f6f6b0e687aa/roles/9399fac5b71b46f79d1ebe5142fc1176 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:40:06.813668 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28864ff-a175-4f6c-bcef-0b4ad9bc2712 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:06.827765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e64ce7-bee7-48f0-aa3b-67c02b75832b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:06.872553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deafe2e-4bd5-43b0-8f77-3aebde20a839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:06.873250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deafe2e-4bd5-43b0-8f77-3aebde20a839 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:06.873539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deafe2e-4bd5-43b0-8f77-3aebde20a839 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:06.873815 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deafe2e-4bd5-43b0-8f77-3aebde20a839 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:06.874599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deafe2e-4bd5-43b0-8f77-3aebde20a839 None admin] RBAC: Adding query filter params (name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:06.874946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deafe2e-4bd5-43b0-8f77-3aebde20a839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:06.876373 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deafe2e-4bd5-43b0-8f77-3aebde20a839 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:06.886851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 318/644] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:40:06 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:06.887793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deafe2e-4bd5-43b0-8f77-3aebde20a839 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:06.915833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:06.957661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28864ff-a175-4f6c-bcef-0b4ad9bc2712 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:06.961437 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:07.012476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a92843-30e6-49dd-8c8f-a61883ccbaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:07.013374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a92843-30e6-49dd-8c8f-a61883ccbaf5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:07.013658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a92843-30e6-49dd-8c8f-a61883ccbaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:07.013933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a92843-30e6-49dd-8c8f-a61883ccbaf5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:07.020407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deafe2e-4bd5-43b0-8f77-3aebde20a839 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:07.026919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a92843-30e6-49dd-8c8f-a61883ccbaf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-39a92843-30e6-49dd-8c8f-a61883ccbaf5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:07.034938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:07.035921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:07.036933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 327/645] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:40:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:07.037664 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a92843-30e6-49dd-8c8f-a61883ccbaf5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:07.086690 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2efca9-51a2-4f6a-ae87-82e65be6295b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:07.087586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2efca9-51a2-4f6a-ae87-82e65be6295b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:07.088077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2efca9-51a2-4f6a-ae87-82e65be6295b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:07.088358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2efca9-51a2-4f6a-ae87-82e65be6295b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:07.165332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a92843-30e6-49dd-8c8f-a61883ccbaf5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:07.177234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2efca9-51a2-4f6a-ae87-82e65be6295b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:07.179064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2efca9-51a2-4f6a-ae87-82e65be6295b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:07.203298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:07.216343 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4eb2e-bb4f-4913-b365-83c8e5e34332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:07.216343 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4eb2e-bb4f-4913-b365-83c8e5e34332 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:07.216343 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4eb2e-bb4f-4913-b365-83c8e5e34332 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:07.216343 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4eb2e-bb4f-4913-b365-83c8e5e34332 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:07.216739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4eb2e-bb4f-4913-b365-83c8e5e34332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:07.216739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4eb2e-bb4f-4913-b365-83c8e5e34332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:07.216739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4eb2e-bb4f-4913-b365-83c8e5e34332 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:07.225077 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 328/646] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:07.225928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4eb2e-bb4f-4913-b365-83c8e5e34332 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:07.253765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:07.306057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 319/647] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:06 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:40:07.307338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2efca9-51a2-4f6a-ae87-82e65be6295b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:07.350474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4eb2e-bb4f-4913-b365-83c8e5e34332 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:07.391713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353e6e-cff2-4a12-8c7b-4ae25c4d3c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:07.392724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353e6e-cff2-4a12-8c7b-4ae25c4d3c3e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:07.393006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353e6e-cff2-4a12-8c7b-4ae25c4d3c3e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:07.393483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353e6e-cff2-4a12-8c7b-4ae25c4d3c3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:07.408916 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3353e6e-cff2-4a12-8c7b-4ae25c4d3c3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:07.410527 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3353e6e-cff2-4a12-8c7b-4ae25c4d3c3e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:07.422619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2efca9-51a2-4f6a-ae87-82e65be6295b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e3353e6e-cff2-4a12-8c7b-4ae25c4d3c3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:07.430703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:07.431650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:07.437093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:07.437093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:07.437093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:07.437093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:07.437093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 329/648] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:07.437093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353e6e-cff2-4a12-8c7b-4ae25c4d3c3e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:07.450186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:07.464198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76878f9-3bff-467a-a274-7d2a00dc9543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:07.464861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76878f9-3bff-467a-a274-7d2a00dc9543 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:07.465136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76878f9-3bff-467a-a274-7d2a00dc9543 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:07.465402 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76878f9-3bff-467a-a274-7d2a00dc9543 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:07.466189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76878f9-3bff-467a-a274-7d2a00dc9543 None admin] RBAC: Adding query filter params (name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:07.466538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76878f9-3bff-467a-a274-7d2a00dc9543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:07.467975 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76878f9-3bff-467a-a274-7d2a00dc9543 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:07.478731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 320/649] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:40:07 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:07.480090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76878f9-3bff-467a-a274-7d2a00dc9543 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:07.529057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:07.557038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353e6e-cff2-4a12-8c7b-4ae25c4d3c3e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:07.598426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdb7e7-40ce-49d3-8299-344a68aab3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:07.599258 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdb7e7-40ce-49d3-8299-344a68aab3c0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:07.599541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdb7e7-40ce-49d3-8299-344a68aab3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:07.599816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdb7e7-40ce-49d3-8299-344a68aab3c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:07.606408 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76878f9-3bff-467a-a274-7d2a00dc9543 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:07.607016 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdb7e7-40ce-49d3-8299-344a68aab3c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:07.608465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdb7e7-40ce-49d3-8299-344a68aab3c0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-a1cdb7e7-40ce-49d3-8299-344a68aab3c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:07.623020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:07.624104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:07.625263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:07.625263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:07.625263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:07.625263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:07.625263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:07.625263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:07.625263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:07.625263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:07.629331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 330/650] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:07.630133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdb7e7-40ce-49d3-8299-344a68aab3c0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:07.652889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe780ff2-d589-4d7f-ac2e-9052a76bf831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:07.653697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe780ff2-d589-4d7f-ac2e-9052a76bf831 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:07.653977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe780ff2-d589-4d7f-ac2e-9052a76bf831 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:07.654250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe780ff2-d589-4d7f-ac2e-9052a76bf831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:07.665842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe780ff2-d589-4d7f-ac2e-9052a76bf831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:07.667569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe780ff2-d589-4d7f-ac2e-9052a76bf831 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-fe780ff2-d589-4d7f-ac2e-9052a76bf831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:07.685719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:07.686915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:07.687819 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:07.687819 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:07.687819 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:07.687819 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:07.687819 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 321/651] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:07.687819 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe780ff2-d589-4d7f-ac2e-9052a76bf831 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:07.729362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:07.758501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdb7e7-40ce-49d3-8299-344a68aab3c0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:07.801355 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe780ff2-d589-4d7f-ac2e-9052a76bf831 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:07.806191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1cef5-e14b-47fa-af5d-d39e6ae1e747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:07.806886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1cef5-e14b-47fa-af5d-d39e6ae1e747 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:07.807174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1cef5-e14b-47fa-af5d-d39e6ae1e747 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:07.807501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1cef5-e14b-47fa-af5d-d39e6ae1e747 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:07.808226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1cef5-e14b-47fa-af5d-d39e6ae1e747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:07.808533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1cef5-e14b-47fa-af5d-d39e6ae1e747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:07.810338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1cef5-e14b-47fa-af5d-d39e6ae1e747 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:07.813918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:07.824217 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 331/652] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:07.825041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1cef5-e14b-47fa-af5d-d39e6ae1e747 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:07.842773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511d963-5d4a-4d2a-a315-cf06c4719ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:07.843736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511d963-5d4a-4d2a-a315-cf06c4719ece None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:07.844398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511d963-5d4a-4d2a-a315-cf06c4719ece None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:07.844531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511d963-5d4a-4d2a-a315-cf06c4719ece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:07.858361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511d963-5d4a-4d2a-a315-cf06c4719ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:07.860011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511d963-5d4a-4d2a-a315-cf06c4719ece None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-2511d963-5d4a-4d2a-a315-cf06c4719ece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:07.883230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:07.884230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:07.889921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:07.889921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:07.889921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:07.889921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:07.889921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 322/653] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:07.889921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511d963-5d4a-4d2a-a315-cf06c4719ece None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:07.944777 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1cef5-e14b-47fa-af5d-d39e6ae1e747 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:07.988262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa76dfd-bd00-4b7b-bcd4-7cd9aeee2b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:07.988856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa76dfd-bd00-4b7b-bcd4-7cd9aeee2b0c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:07.989363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa76dfd-bd00-4b7b-bcd4-7cd9aeee2b0c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:07.989363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa76dfd-bd00-4b7b-bcd4-7cd9aeee2b0c None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:07.990167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa76dfd-bd00-4b7b-bcd4-7cd9aeee2b0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:07.990475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa76dfd-bd00-4b7b-bcd4-7cd9aeee2b0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:07.991963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa76dfd-bd00-4b7b-bcd4-7cd9aeee2b0c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:07.992821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:08.010272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 332/654] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:08.010272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa76dfd-bd00-4b7b-bcd4-7cd9aeee2b0c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:08.010552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511d963-5d4a-4d2a-a315-cf06c4719ece None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:08.044540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc94d5-4d9a-432b-b159-8e17ebc3b254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:08.045220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc94d5-4d9a-432b-b159-8e17ebc3b254 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:08.045477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc94d5-4d9a-432b-b159-8e17ebc3b254 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:08.045748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc94d5-4d9a-432b-b159-8e17ebc3b254 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:08.046463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc94d5-4d9a-432b-b159-8e17ebc3b254 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:08.046765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc94d5-4d9a-432b-b159-8e17ebc3b254 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:08.048314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc94d5-4d9a-432b-b159-8e17ebc3b254 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:08.067705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 323/655] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:08.067705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc94d5-4d9a-432b-b159-8e17ebc3b254 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:08.098607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:08.154633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa76dfd-bd00-4b7b-bcd4-7cd9aeee2b0c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:08.204932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1157d-5b60-414d-b6d2-29a7ba83d20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:08.205774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1157d-5b60-414d-b6d2-29a7ba83d20f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:08.206052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1157d-5b60-414d-b6d2-29a7ba83d20f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:08.206240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc94d5-4d9a-432b-b159-8e17ebc3b254 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:08.206341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1157d-5b60-414d-b6d2-29a7ba83d20f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:08.226714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1157d-5b60-414d-b6d2-29a7ba83d20f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:08.229588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1157d-5b60-414d-b6d2-29a7ba83d20f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-2cd1157d-5b60-414d-b6d2-29a7ba83d20f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:08.246687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:08.247704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:08.248648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:08.248648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:08.248648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:08.248648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:08.248648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:08.248648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:08.248648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:08.248648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:08.248648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 17:40:08.248648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:08.251101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 333/656] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:40:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:08.252543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1157d-5b60-414d-b6d2-29a7ba83d20f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:08.261273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b23ead-aeb3-4b89-b9e6-0a6b058e56b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:08.261948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b23ead-aeb3-4b89-b9e6-0a6b058e56b5 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:08.262231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b23ead-aeb3-4b89-b9e6-0a6b058e56b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:08.262504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b23ead-aeb3-4b89-b9e6-0a6b058e56b5 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:08.263264 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b23ead-aeb3-4b89-b9e6-0a6b058e56b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:08.263603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b23ead-aeb3-4b89-b9e6-0a6b058e56b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:08.264938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b23ead-aeb3-4b89-b9e6-0a6b058e56b5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:08.282187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 324/657] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:08.282187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b23ead-aeb3-4b89-b9e6-0a6b058e56b5 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:08.289744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:08.372942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1157d-5b60-414d-b6d2-29a7ba83d20f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:08.389529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:08.409388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b23ead-aeb3-4b89-b9e6-0a6b058e56b5 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:08.412893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6b291-2e4a-4c63-a1ed-bbae43518abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:08.413554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6b291-2e4a-4c63-a1ed-bbae43518abc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:08.413832 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6b291-2e4a-4c63-a1ed-bbae43518abc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:08.414151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6b291-2e4a-4c63-a1ed-bbae43518abc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:08.415022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6b291-2e4a-4c63-a1ed-bbae43518abc None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=683c54f489f440b9842cb94980214df1) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:08.415331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6b291-2e4a-4c63-a1ed-bbae43518abc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:08.417018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6b291-2e4a-4c63-a1ed-bbae43518abc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:08.434852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 334/658] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:08 2022] GET /identity/v3/role_assignments?user.id=683c54f489f440b9842cb94980214df1&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 610 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:08.434852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6b291-2e4a-4c63-a1ed-bbae43518abc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:08.464079 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54078628-755f-41b2-b9d6-f3d1e690aadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:08.465004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54078628-755f-41b2-b9d6-f3d1e690aadc None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:08.465353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54078628-755f-41b2-b9d6-f3d1e690aadc None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:08.465694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54078628-755f-41b2-b9d6-f3d1e690aadc None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:08.475744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54078628-755f-41b2-b9d6-f3d1e690aadc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:08.477239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54078628-755f-41b2-b9d6-f3d1e690aadc None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-54078628-755f-41b2-b9d6-f3d1e690aadc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:08.494059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:08.495296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:08.496281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:08.496281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:08.496281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:08.496281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:08.496281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:08.496281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:08.496281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:08.496281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:08.496281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 325/659] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:08.496281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54078628-755f-41b2-b9d6-f3d1e690aadc None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:08.576165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:08.591991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6b291-2e4a-4c63-a1ed-bbae43518abc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:08.619161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54078628-755f-41b2-b9d6-f3d1e690aadc None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:08.644242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab04bf-0689-4a8a-93f3-0214c4172bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:08.645490 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab04bf-0689-4a8a-93f3-0214c4172bbf None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:08.645991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab04bf-0689-4a8a-93f3-0214c4172bbf None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:08.646533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab04bf-0689-4a8a-93f3-0214c4172bbf None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:08.647622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab04bf-0689-4a8a-93f3-0214c4172bbf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:08.648142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab04bf-0689-4a8a-93f3-0214c4172bbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:08.650035 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab04bf-0689-4a8a-93f3-0214c4172bbf None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:08.657655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86276be1-88eb-4b54-976d-1d47131bd6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:08.658909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86276be1-88eb-4b54-976d-1d47131bd6cf None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:08.659200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86276be1-88eb-4b54-976d-1d47131bd6cf None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:08.659475 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86276be1-88eb-4b54-976d-1d47131bd6cf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:08.668529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86276be1-88eb-4b54-976d-1d47131bd6cf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:08.670311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86276be1-88eb-4b54-976d-1d47131bd6cf None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:08.675154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 335/660] 10.222.0.121 () {56 vars in 1144 bytes} [Thu May 26 17:40:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:08.676355 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab04bf-0689-4a8a-93f3-0214c4172bbf None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-86276be1-88eb-4b54-976d-1d47131bd6cf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:08.691263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:08.692269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:08.693419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:08.693419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:08.693419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:08.693419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:08.693419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:08.693419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:08.693419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:08.693419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:08.693419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:40:08.693419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:08.693419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 326/661] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:40:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:08.694020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86276be1-88eb-4b54-976d-1d47131bd6cf None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:08.730521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:08.809853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab04bf-0689-4a8a-93f3-0214c4172bbf None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:08.823633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86276be1-88eb-4b54-976d-1d47131bd6cf None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:08.862175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a7a7d-cd68-4b35-bbdf-d2a11ae9953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:08.862736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a7a7d-cd68-4b35-bbdf-d2a11ae9953a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:08.863039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a7a7d-cd68-4b35-bbdf-d2a11ae9953a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:08.863309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a7a7d-cd68-4b35-bbdf-d2a11ae9953a None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:08.864097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a7a7d-cd68-4b35-bbdf-d2a11ae9953a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:08.864392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a7a7d-cd68-4b35-bbdf-d2a11ae9953a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:08.865828 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a7a7d-cd68-4b35-bbdf-d2a11ae9953a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:08.869359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:08.879859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 336/662] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:08.885603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a7a7d-cd68-4b35-bbdf-d2a11ae9953a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:08.885724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248a0c7-e571-4c2a-b5a8-2737b37e52e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:08.885724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248a0c7-e571-4c2a-b5a8-2737b37e52e5 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:08.885724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248a0c7-e571-4c2a-b5a8-2737b37e52e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:08.885724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248a0c7-e571-4c2a-b5a8-2737b37e52e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:08.984049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248a0c7-e571-4c2a-b5a8-2737b37e52e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:08.986862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248a0c7-e571-4c2a-b5a8-2737b37e52e5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:09.003261 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a7a7d-cd68-4b35-bbdf-d2a11ae9953a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:09.019365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:09.040686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3445a-26a9-4807-859c-40ec3343257a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:09.041978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3445a-26a9-4807-859c-40ec3343257a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:09.042978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3445a-26a9-4807-859c-40ec3343257a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:09.043495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3445a-26a9-4807-859c-40ec3343257a None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:09.045144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3445a-26a9-4807-859c-40ec3343257a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:09.045778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3445a-26a9-4807-859c-40ec3343257a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:09.047924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3445a-26a9-4807-859c-40ec3343257a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:09.077408 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 337/663] 10.222.0.121 () {56 vars in 1142 bytes} [Thu May 26 17:40:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:09.077408 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3445a-26a9-4807-859c-40ec3343257a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:09.138985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 327/664] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:08 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:40:09.139905 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248a0c7-e571-4c2a-b5a8-2737b37e52e5 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:09.172135 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:09.187513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3445a-26a9-4807-859c-40ec3343257a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:09.229689 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294733b4-5c50-4e26-9176-2a856a215cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:09.230356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294733b4-5c50-4e26-9176-2a856a215cd6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:09.230639 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294733b4-5c50-4e26-9176-2a856a215cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:09.230937 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294733b4-5c50-4e26-9176-2a856a215cd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:09.231789 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294733b4-5c50-4e26-9176-2a856a215cd6 None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=c0889a69a018452d8de1924072739d17) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:09.232100 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294733b4-5c50-4e26-9176-2a856a215cd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:09.233550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294733b4-5c50-4e26-9176-2a856a215cd6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:09.247248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 338/665] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:09 2022] GET /identity/v3/role_assignments?user.id=c0889a69a018452d8de1924072739d17&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 610 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:09.250274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc444d1-cc89-4d21-af9c-6e83ada1fd23 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:09.250919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc444d1-cc89-4d21-af9c-6e83ada1fd23 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:09.251808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc444d1-cc89-4d21-af9c-6e83ada1fd23 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:09.259493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 339/666] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:09 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:09.261664 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc444d1-cc89-4d21-af9c-6e83ada1fd23 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:09.263679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248a0c7-e571-4c2a-b5a8-2737b37e52e5 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:09.294111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:09.309049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d0800-65f5-4834-bc27-e181450581f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:09.309725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d0800-65f5-4834-bc27-e181450581f9 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:09.310008 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d0800-65f5-4834-bc27-e181450581f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:09.310282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d0800-65f5-4834-bc27-e181450581f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:09.409032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc444d1-cc89-4d21-af9c-6e83ada1fd23 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:09.411267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d0800-65f5-4834-bc27-e181450581f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:09.414122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d0800-65f5-4834-bc27-e181450581f9 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:09.453329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:09.458293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ecb5c-eb07-456d-8959-420d95ebdf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:09.459077 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ecb5c-eb07-456d-8959-420d95ebdf46 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:09.459362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ecb5c-eb07-456d-8959-420d95ebdf46 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:09.459639 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ecb5c-eb07-456d-8959-420d95ebdf46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:09.473083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ecb5c-eb07-456d-8959-420d95ebdf46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:09.474777 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ecb5c-eb07-456d-8959-420d95ebdf46 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-de8ecb5c-eb07-456d-8959-420d95ebdf46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:09.494318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:09.495959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:09.496833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:09.496833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:09.496833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:09.496833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:09.500833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 340/667] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:09.502594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895c2f3-5718-4613-91e2-6dbbe9b5da27 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:09.503018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895c2f3-5718-4613-91e2-6dbbe9b5da27 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:09.503232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895c2f3-5718-4613-91e2-6dbbe9b5da27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:09.526390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1895c2f3-5718-4613-91e2-6dbbe9b5da27 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:09.527652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1895c2f3-5718-4613-91e2-6dbbe9b5da27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:09.574149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 328/668] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:40:09.575413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d0800-65f5-4834-bc27-e181450581f9 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:09.621995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 341/669] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:40:09.622952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895c2f3-5718-4613-91e2-6dbbe9b5da27 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:09.625450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:09.715347 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d0800-65f5-4834-bc27-e181450581f9 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:09.750586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:09.762641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4ae1c-036c-4dae-bb22-0d3d5672d5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:09.763569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4ae1c-036c-4dae-bb22-0d3d5672d5bb None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:09.763925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4ae1c-036c-4dae-bb22-0d3d5672d5bb None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:09.764266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4ae1c-036c-4dae-bb22-0d3d5672d5bb None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:09.765353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4ae1c-036c-4dae-bb22-0d3d5672d5bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:09.765731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4ae1c-036c-4dae-bb22-0d3d5672d5bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:09.767387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4ae1c-036c-4dae-bb22-0d3d5672d5bb None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:09.785125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 329/670] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:09.786017 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895c2f3-5718-4613-91e2-6dbbe9b5da27 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:09.786134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4ae1c-036c-4dae-bb22-0d3d5672d5bb None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:09.832679 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf61d4a-d768-4c5f-b3b2-987b8bd40081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:09.833607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf61d4a-d768-4c5f-b3b2-987b8bd40081 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:09.833953 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf61d4a-d768-4c5f-b3b2-987b8bd40081 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:09.834294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf61d4a-d768-4c5f-b3b2-987b8bd40081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:09.851459 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf61d4a-d768-4c5f-b3b2-987b8bd40081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:09.853032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf61d4a-d768-4c5f-b3b2-987b8bd40081 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-abf61d4a-d768-4c5f-b3b2-987b8bd40081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:09.875177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:09.876146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:09.877744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:09.877744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:09.877744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:09.877744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:09.881015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 342/671] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:09.881979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf61d4a-d768-4c5f-b3b2-987b8bd40081 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:09.911264 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4ae1c-036c-4dae-bb22-0d3d5672d5bb None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:09.951240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5555b-f999-4c2c-b432-a2144b78ebfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:09.952587 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5555b-f999-4c2c-b432-a2144b78ebfb None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:09.953021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5555b-f999-4c2c-b432-a2144b78ebfb None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:09.953478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5555b-f999-4c2c-b432-a2144b78ebfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:09.962564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:09.973811 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5555b-f999-4c2c-b432-a2144b78ebfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-e5a5555b-f999-4c2c-b432-a2144b78ebfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:09.983972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:09.984966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:09.986924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:09.986924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:09.986924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:09.986924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:09.986924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:09.986924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:09.986924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:09.986924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:09.986924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:09.986924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:09.986924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:09.986924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:09.991960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 330/672] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:40:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:09.992869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5555b-f999-4c2c-b432-a2144b78ebfb None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:10.037788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf61d4a-d768-4c5f-b3b2-987b8bd40081 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:10.058561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:10.088677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5396af5-1bd1-4b9d-835d-072b23902a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:10.090045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5396af5-1bd1-4b9d-835d-072b23902a73 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:10.091192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5396af5-1bd1-4b9d-835d-072b23902a73 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:10.091471 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5396af5-1bd1-4b9d-835d-072b23902a73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:10.114658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5396af5-1bd1-4b9d-835d-072b23902a73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:10.117318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5396af5-1bd1-4b9d-835d-072b23902a73 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:10.129631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5555b-f999-4c2c-b432-a2144b78ebfb None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-b5396af5-1bd1-4b9d-835d-072b23902a73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:10.136391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:10.137441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:10.138534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:10.138534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:10.138534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:10.138534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:10.138534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:10.138534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:10.138534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:10.138534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:10.141145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 343/673] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:10.143242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5396af5-1bd1-4b9d-835d-072b23902a73 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:10.188665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c054b83-d064-4f46-8c6e-b7c1c0e0e3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:10.189509 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c054b83-d064-4f46-8c6e-b7c1c0e0e3d0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:10.189816 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c054b83-d064-4f46-8c6e-b7c1c0e0e3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:10.190175 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c054b83-d064-4f46-8c6e-b7c1c0e0e3d0 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:10.191171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c054b83-d064-4f46-8c6e-b7c1c0e0e3d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:10.191481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c054b83-d064-4f46-8c6e-b7c1c0e0e3d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:10.193992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c054b83-d064-4f46-8c6e-b7c1c0e0e3d0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:10.212367 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 331/674] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:10.213353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c054b83-d064-4f46-8c6e-b7c1c0e0e3d0 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:10.304363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5396af5-1bd1-4b9d-835d-072b23902a73 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:10.321942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:10.348718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33bae1-7fe9-42f7-9425-a2fd39983970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:10.349651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33bae1-7fe9-42f7-9425-a2fd39983970 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:10.350111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33bae1-7fe9-42f7-9425-a2fd39983970 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:10.350883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33bae1-7fe9-42f7-9425-a2fd39983970 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:10.352457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33bae1-7fe9-42f7-9425-a2fd39983970 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:10.352917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33bae1-7fe9-42f7-9425-a2fd39983970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:10.354741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33bae1-7fe9-42f7-9425-a2fd39983970 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:10.365424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c054b83-d064-4f46-8c6e-b7c1c0e0e3d0 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:10.368426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 344/675] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:40:10 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:10.369575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33bae1-7fe9-42f7-9425-a2fd39983970 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:10.386678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:10.428733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550724c4-4354-4edb-a078-847463b84d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:10.429417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550724c4-4354-4edb-a078-847463b84d50 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:10.429699 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550724c4-4354-4edb-a078-847463b84d50 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:10.430399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550724c4-4354-4edb-a078-847463b84d50 None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:10.430770 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550724c4-4354-4edb-a078-847463b84d50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:10.431112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550724c4-4354-4edb-a078-847463b84d50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:10.432559 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550724c4-4354-4edb-a078-847463b84d50 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:10.452459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 332/676] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:10.452459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550724c4-4354-4edb-a078-847463b84d50 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:10.512239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33bae1-7fe9-42f7-9425-a2fd39983970 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:10.556592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d881f-7c39-46aa-898d-d78a5bfe09ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:10.557727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d881f-7c39-46aa-898d-d78a5bfe09ed None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:10.558882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d881f-7c39-46aa-898d-d78a5bfe09ed None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:10.559417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d881f-7c39-46aa-898d-d78a5bfe09ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:10.570114 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d881f-7c39-46aa-898d-d78a5bfe09ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:10.572439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d881f-7c39-46aa-898d-d78a5bfe09ed None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-136d881f-7c39-46aa-898d-d78a5bfe09ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:10.591990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:10.592972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:10.593869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:10.593869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:10.593869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:10.593869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:10.593869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:10.593869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:10.593869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:10.593869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:10.598858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 345/677] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:10.599773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d881f-7c39-46aa-898d-d78a5bfe09ed None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:10.601528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550724c4-4354-4edb-a078-847463b84d50 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:10.640831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:10.650268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22b0b4-bb47-48c0-a34f-1413cfa17ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:10.665896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22b0b4-bb47-48c0-a34f-1413cfa17ac8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:10.665896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22b0b4-bb47-48c0-a34f-1413cfa17ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:10.665896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22b0b4-bb47-48c0-a34f-1413cfa17ac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:10.668723 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22b0b4-bb47-48c0-a34f-1413cfa17ac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:10.670597 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22b0b4-bb47-48c0-a34f-1413cfa17ac8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-cc22b0b4-bb47-48c0-a34f-1413cfa17ac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:10.692930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:10.693911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:10.694794 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:10.694794 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:10.694794 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:10.694794 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:10.694794 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 333/678] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:10.695513 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22b0b4-bb47-48c0-a34f-1413cfa17ac8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:10.715789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:10.730675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d881f-7c39-46aa-898d-d78a5bfe09ed None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:10.774306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ecf28-ae23-4cbd-a330-cd1e82060ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:10.775305 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ecf28-ae23-4cbd-a330-cd1e82060ab4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:10.775688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ecf28-ae23-4cbd-a330-cd1e82060ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:10.776138 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ecf28-ae23-4cbd-a330-cd1e82060ab4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:10.777324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ecf28-ae23-4cbd-a330-cd1e82060ab4 None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=08709b462300437598c8db331e5ad20f) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:10.777632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ecf28-ae23-4cbd-a330-cd1e82060ab4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:10.779327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ecf28-ae23-4cbd-a330-cd1e82060ab4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:10.794850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 346/679] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:10 2022] GET /identity/v3/role_assignments?user.id=08709b462300437598c8db331e5ad20f&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 610 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:10.794850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ecf28-ae23-4cbd-a330-cd1e82060ab4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:10.856064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22b0b4-bb47-48c0-a34f-1413cfa17ac8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:10.942188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0040d-4a1b-4c8d-9e78-fa8cab5b7e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:10.942891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0040d-4a1b-4c8d-9e78-fa8cab5b7e73 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:10.943176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0040d-4a1b-4c8d-9e78-fa8cab5b7e73 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:10.943446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0040d-4a1b-4c8d-9e78-fa8cab5b7e73 None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:10.944242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0040d-4a1b-4c8d-9e78-fa8cab5b7e73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:10.944548 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0040d-4a1b-4c8d-9e78-fa8cab5b7e73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:10.945992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0040d-4a1b-4c8d-9e78-fa8cab5b7e73 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:10.960175 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 334/680] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:10.968778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0040d-4a1b-4c8d-9e78-fa8cab5b7e73 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:11.010608 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:11.013159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ecf28-ae23-4cbd-a330-cd1e82060ab4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:11.097896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db21f13-d2a6-43a6-8a05-09937bd50067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:11.098507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db21f13-d2a6-43a6-8a05-09937bd50067 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:11.098784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db21f13-d2a6-43a6-8a05-09937bd50067 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:11.099646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db21f13-d2a6-43a6-8a05-09937bd50067 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:11.099892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db21f13-d2a6-43a6-8a05-09937bd50067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:11.100195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db21f13-d2a6-43a6-8a05-09937bd50067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:11.102171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db21f13-d2a6-43a6-8a05-09937bd50067 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:11.115032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 347/681] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:11.120681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db21f13-d2a6-43a6-8a05-09937bd50067 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:11.132664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:11.159162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0040d-4a1b-4c8d-9e78-fa8cab5b7e73 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:11.211933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af97de9-51a0-461f-b2a6-ac416601b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:11.213041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af97de9-51a0-461f-b2a6-ac416601b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:11.213331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af97de9-51a0-461f-b2a6-ac416601b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:11.213609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af97de9-51a0-461f-b2a6-ac416601b3e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:11.214730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af97de9-51a0-461f-b2a6-ac416601b3e0 None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=25d85d02abed40199dd2515ecf84b7fb) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:11.215078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af97de9-51a0-461f-b2a6-ac416601b3e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:11.217201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af97de9-51a0-461f-b2a6-ac416601b3e0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:11.226974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 335/682] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:10 2022] GET /identity/v3/role_assignments?user.id=25d85d02abed40199dd2515ecf84b7fb&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 610 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:11.228341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e654d-9756-445e-bf53-aeb420d05bbf None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:11.228626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e654d-9756-445e-bf53-aeb420d05bbf None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:11.228976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e654d-9756-445e-bf53-aeb420d05bbf None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:11.231406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 336/683] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:11 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:11.238893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08633a7d-cec5-48ed-b627-35ba8362e915 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:11.239241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08633a7d-cec5-48ed-b627-35ba8362e915 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:11.239467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08633a7d-cec5-48ed-b627-35ba8362e915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:11.263815 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-08633a7d-cec5-48ed-b627-35ba8362e915 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:11.265432 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-08633a7d-cec5-48ed-b627-35ba8362e915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:11.271687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db21f13-d2a6-43a6-8a05-09937bd50067 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:11.317842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd403c6-8093-472c-90f4-a83fb99a628f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:11.319096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd403c6-8093-472c-90f4-a83fb99a628f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:11.319591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd403c6-8093-472c-90f4-a83fb99a628f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:11.320101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd403c6-8093-472c-90f4-a83fb99a628f None admin] PATH_INFO: `/v3/users/swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:11.330153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd403c6-8093-472c-90f4-a83fb99a628f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:11.332121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd403c6-8093-472c-90f4-a83fb99a628f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-dcd403c6-8093-472c-90f4-a83fb99a628f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:11.348723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:11.349745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:11.350687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:11.350687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:11.350687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:11.350687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:11.350687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:11.350687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:11.350687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:11.350687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:11.350687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 17:40:11.350687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:11.353502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 348/684] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:40:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:11.358672 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd403c6-8093-472c-90f4-a83fb99a628f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:11.359739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 337/685] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:40:11.387061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08633a7d-cec5-48ed-b627-35ba8362e915 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:11.425949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:11.503086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:11.541841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08633a7d-cec5-48ed-b627-35ba8362e915 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:11.601454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f75a5-c42d-465b-8ae5-16b604c26489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:11.602611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f75a5-c42d-465b-8ae5-16b604c26489 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:11.603110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f75a5-c42d-465b-8ae5-16b604c26489 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:11.603392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd403c6-8093-472c-90f4-a83fb99a628f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:11.603696 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f75a5-c42d-465b-8ae5-16b604c26489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:11.625083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f75a5-c42d-465b-8ae5-16b604c26489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:11.628131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f75a5-c42d-465b-8ae5-16b604c26489 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-c42f75a5-c42d-465b-8ae5-16b604c26489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:11.662511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:11.663535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:11.664412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:11.664412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:11.664412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:11.664412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:11.664878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 338/686] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:11.673723 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f75a5-c42d-465b-8ae5-16b604c26489 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:11.674684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f022e-5a1f-4dcd-ac3b-6ce0cedfa5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:11.676087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f022e-5a1f-4dcd-ac3b-6ce0cedfa5db None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:11.676580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f022e-5a1f-4dcd-ac3b-6ce0cedfa5db None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:11.677030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f022e-5a1f-4dcd-ac3b-6ce0cedfa5db None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:11.678095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f022e-5a1f-4dcd-ac3b-6ce0cedfa5db None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:11.678490 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f022e-5a1f-4dcd-ac3b-6ce0cedfa5db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:11.680133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f022e-5a1f-4dcd-ac3b-6ce0cedfa5db None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:11.702308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 349/687] 10.222.0.121 () {56 vars in 1140 bytes} [Thu May 26 17:40:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:11.708729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f022e-5a1f-4dcd-ac3b-6ce0cedfa5db None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:11.856460 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f75a5-c42d-465b-8ae5-16b604c26489 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:11.884856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:11.890184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:11.903029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645de49b-a9e0-41a2-abf2-38b714f42329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:11.903959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645de49b-a9e0-41a2-abf2-38b714f42329 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:11.904273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645de49b-a9e0-41a2-abf2-38b714f42329 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:11.904551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645de49b-a9e0-41a2-abf2-38b714f42329 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:11.905292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645de49b-a9e0-41a2-abf2-38b714f42329 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:11.905670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645de49b-a9e0-41a2-abf2-38b714f42329 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:11.907235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645de49b-a9e0-41a2-abf2-38b714f42329 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:11.911161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f022e-5a1f-4dcd-ac3b-6ce0cedfa5db None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:11.926304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 339/688] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:11.927848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dde154-8103-4aa5-86b6-f7bd10dabccd None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:11.928137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dde154-8103-4aa5-86b6-f7bd10dabccd None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:11.928413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dde154-8103-4aa5-86b6-f7bd10dabccd None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:11.930118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 340/689] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:11.931254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dde154-8103-4aa5-86b6-f7bd10dabccd None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:11.955853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87267828-b20d-4468-bd62-4c0bed7e2c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:11.956838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87267828-b20d-4468-bd62-4c0bed7e2c42 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:11.957225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87267828-b20d-4468-bd62-4c0bed7e2c42 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:11.957665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87267828-b20d-4468-bd62-4c0bed7e2c42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:12.036378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87267828-b20d-4468-bd62-4c0bed7e2c42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:12.038003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87267828-b20d-4468-bd62-4c0bed7e2c42 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:12.050612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dde154-8103-4aa5-86b6-f7bd10dabccd None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:12.098441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667846b6-cf3e-415d-bc96-a6e9fd8a2a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:12.099257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667846b6-cf3e-415d-bc96-a6e9fd8a2a41 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:12.099631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667846b6-cf3e-415d-bc96-a6e9fd8a2a41 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:12.099980 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667846b6-cf3e-415d-bc96-a6e9fd8a2a41 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:12.100998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667846b6-cf3e-415d-bc96-a6e9fd8a2a41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:12.107782 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667846b6-cf3e-415d-bc96-a6e9fd8a2a41 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:12.144147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:12.175733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 350/690] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:11 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:40:12.177419 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81edb5-a564-4cc7-be0a-fecd56adbf7d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:12.177840 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81edb5-a564-4cc7-be0a-fecd56adbf7d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:12.178271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81edb5-a564-4cc7-be0a-fecd56adbf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:12.200824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-da81edb5-a564-4cc7-be0a-fecd56adbf7d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:12.203311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-da81edb5-a564-4cc7-be0a-fecd56adbf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:12.216099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 341/691] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:40:11 2022] POST /identity/v3/users => generated 255 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:40:12.217526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667846b6-cf3e-415d-bc96-a6e9fd8a2a41 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:12.294547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 351/692] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:40:12.319952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81edb5-a564-4cc7-be0a-fecd56adbf7d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:12.330511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:12.447589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667846b6-cf3e-415d-bc96-a6e9fd8a2a41 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:12.461785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81edb5-a564-4cc7-be0a-fecd56adbf7d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:12.475373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:12.517922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe10a43-ea52-404c-a335-ae8deec38b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:12.518810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe10a43-ea52-404c-a335-ae8deec38b71 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:12.519116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe10a43-ea52-404c-a335-ae8deec38b71 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:12.519389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe10a43-ea52-404c-a335-ae8deec38b71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:12.524823 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631d4fa-daca-4a79-a8a0-b2c73f43ade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:12.525641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631d4fa-daca-4a79-a8a0-b2c73f43ade3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:12.525926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631d4fa-daca-4a79-a8a0-b2c73f43ade3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:12.526201 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631d4fa-daca-4a79-a8a0-b2c73f43ade3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:12.541295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe10a43-ea52-404c-a335-ae8deec38b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:12.543320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe10a43-ea52-404c-a335-ae8deec38b71 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:12.547481 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631d4fa-daca-4a79-a8a0-b2c73f43ade3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-2631d4fa-daca-4a79-a8a0-b2c73f43ade3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:12.556293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:12.557254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:12.558306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 352/693] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:40:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:12.563678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631d4fa-daca-4a79-a8a0-b2c73f43ade3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-5fe10a43-ea52-404c-a335-ae8deec38b71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:12.572760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:12.574566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:12.576810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:12.576810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:12.576810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:12.576810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:12.576810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 342/694] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:12.583895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe10a43-ea52-404c-a335-ae8deec38b71 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:12.709465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631d4fa-daca-4a79-a8a0-b2c73f43ade3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:12.717577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:12.728087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe10a43-ea52-404c-a335-ae8deec38b71 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:12.750888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90ae62-8854-4c48-a269-d2452a864883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:12.752116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90ae62-8854-4c48-a269-d2452a864883 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:12.752621 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90ae62-8854-4c48-a269-d2452a864883 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:12.753086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90ae62-8854-4c48-a269-d2452a864883 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:12.754010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90ae62-8854-4c48-a269-d2452a864883 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:12.754472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90ae62-8854-4c48-a269-d2452a864883 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:12.756061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90ae62-8854-4c48-a269-d2452a864883 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:12.768251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 353/695] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:40:12 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:12.770503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631d205-06e8-47bf-bc09-b8cd0deea2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:12.771981 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631d205-06e8-47bf-bc09-b8cd0deea2f1 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:12.771981 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631d205-06e8-47bf-bc09-b8cd0deea2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:12.771981 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631d205-06e8-47bf-bc09-b8cd0deea2f1 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:12.772586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631d205-06e8-47bf-bc09-b8cd0deea2f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:12.772894 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631d205-06e8-47bf-bc09-b8cd0deea2f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:12.773895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90ae62-8854-4c48-a269-d2452a864883 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:12.774271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631d205-06e8-47bf-bc09-b8cd0deea2f1 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:12.788074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 343/696] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:12.793814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631d205-06e8-47bf-bc09-b8cd0deea2f1 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:12.853776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:12.918964 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631d205-06e8-47bf-bc09-b8cd0deea2f1 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:12.953691 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e7e79-1402-4c46-8e6e-e545fb3441cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:12.955256 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e7e79-1402-4c46-8e6e-e545fb3441cc None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:12.955886 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e7e79-1402-4c46-8e6e-e545fb3441cc None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:12.956043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e7e79-1402-4c46-8e6e-e545fb3441cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:12.966991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e7e79-1402-4c46-8e6e-e545fb3441cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:12.966991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e7e79-1402-4c46-8e6e-e545fb3441cc None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-844e7e79-1402-4c46-8e6e-e545fb3441cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:12.982010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:12.983136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:12.984341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:12.984341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:12.984341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:12.984341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:12.984341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:12.984341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:12.984341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:12.984341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:12.986855 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 344/697] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:12.987772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90ae62-8854-4c48-a269-d2452a864883 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:12.991449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e7e79-1402-4c46-8e6e-e545fb3441cc None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:13.019467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:13.062370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcbd7c-7c34-441a-b430-bdc4f583b857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:13.063516 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcbd7c-7c34-441a-b430-bdc4f583b857 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:13.064362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcbd7c-7c34-441a-b430-bdc4f583b857 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:13.064963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcbd7c-7c34-441a-b430-bdc4f583b857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:13.083125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcbd7c-7c34-441a-b430-bdc4f583b857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:13.085410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcbd7c-7c34-441a-b430-bdc4f583b857 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-56dcbd7c-7c34-441a-b430-bdc4f583b857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:13.107004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:13.107970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:13.108892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:13.108892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:13.108892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:13.108892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:13.112341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 354/698] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:13.121022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcbd7c-7c34-441a-b430-bdc4f583b857 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:13.205914 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e7e79-1402-4c46-8e6e-e545fb3441cc None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:13.272620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cdecc-2e03-4da9-9d1b-ee2bdccc15bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:13.273202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cdecc-2e03-4da9-9d1b-ee2bdccc15bd None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:13.273489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cdecc-2e03-4da9-9d1b-ee2bdccc15bd None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:13.273767 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cdecc-2e03-4da9-9d1b-ee2bdccc15bd None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:13.274568 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cdecc-2e03-4da9-9d1b-ee2bdccc15bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:13.274903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cdecc-2e03-4da9-9d1b-ee2bdccc15bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:13.276351 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cdecc-2e03-4da9-9d1b-ee2bdccc15bd None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:13.279121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:13.303805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 345/699] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:13.309152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cdecc-2e03-4da9-9d1b-ee2bdccc15bd None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:13.312266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcbd7c-7c34-441a-b430-bdc4f583b857 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:13.365639 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859f7e0-42d5-4bd8-8692-534af5129526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:13.366327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859f7e0-42d5-4bd8-8692-534af5129526 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:13.366596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859f7e0-42d5-4bd8-8692-534af5129526 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:13.366892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859f7e0-42d5-4bd8-8692-534af5129526 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:13.367826 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859f7e0-42d5-4bd8-8692-534af5129526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:13.368135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859f7e0-42d5-4bd8-8692-534af5129526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:13.370001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859f7e0-42d5-4bd8-8692-534af5129526 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:13.386866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 355/700] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:13.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859f7e0-42d5-4bd8-8692-534af5129526 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:13.482959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:13.561386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cdecc-2e03-4da9-9d1b-ee2bdccc15bd None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:13.653431 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf504e6-0e27-43ad-a720-a3af6b68cdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:13.654573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf504e6-0e27-43ad-a720-a3af6b68cdbe None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:13.655090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf504e6-0e27-43ad-a720-a3af6b68cdbe None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:13.655595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf504e6-0e27-43ad-a720-a3af6b68cdbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:13.656677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf504e6-0e27-43ad-a720-a3af6b68cdbe None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=eb9ba9c73956403ba870f6f6b0e687aa) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:13.657206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf504e6-0e27-43ad-a720-a3af6b68cdbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:13.666855 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf504e6-0e27-43ad-a720-a3af6b68cdbe None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:13.667032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859f7e0-42d5-4bd8-8692-534af5129526 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:13.693287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 346/701] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:13 2022] GET /identity/v3/role_assignments?user.id=eb9ba9c73956403ba870f6f6b0e687aa&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 610 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:13.706099 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df7fc6-e30b-49a2-b14e-53256e24bbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:13.707326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df7fc6-e30b-49a2-b14e-53256e24bbfb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:13.707653 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df7fc6-e30b-49a2-b14e-53256e24bbfb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:13.707923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df7fc6-e30b-49a2-b14e-53256e24bbfb None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:13.716007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df7fc6-e30b-49a2-b14e-53256e24bbfb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:13.717465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df7fc6-e30b-49a2-b14e-53256e24bbfb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-d7df7fc6-e30b-49a2-b14e-53256e24bbfb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:13.734071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:13.741357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:13.742776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:13.742776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:13.742776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:13.742776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:13.742776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:13.742776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:13.742776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:13.742776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:13.742776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:40:13.742776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:13.742776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 356/702] 10.222.0.121 () {56 vars in 1098 bytes} [Thu May 26 17:40:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:13.743387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf504e6-0e27-43ad-a720-a3af6b68cdbe None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:13.858733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a67856-9cef-4426-8464-d94e1fbe9bb9 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:13.859088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a67856-9cef-4426-8464-d94e1fbe9bb9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:13.859370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a67856-9cef-4426-8464-d94e1fbe9bb9 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:13.861002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 357/703] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:13.873375 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbcf90-aa0a-45c6-9a96-e7b4f00f22a2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:13.873704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbcf90-aa0a-45c6-9a96-e7b4f00f22a2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:13.873918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbcf90-aa0a-45c6-9a96-e7b4f00f22a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:13.893367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:13.905407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf504e6-0e27-43ad-a720-a3af6b68cdbe None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:13.914239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c6dbcf90-aa0a-45c6-9a96-e7b4f00f22a2 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:13.915529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c6dbcf90-aa0a-45c6-9a96-e7b4f00f22a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:13.959960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6a15a-96b2-4f18-b5e1-351fa48db162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:13.960864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6a15a-96b2-4f18-b5e1-351fa48db162 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:13.961285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6a15a-96b2-4f18-b5e1-351fa48db162 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:13.961769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6a15a-96b2-4f18-b5e1-351fa48db162 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:13.962733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6a15a-96b2-4f18-b5e1-351fa48db162 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:13.963103 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6a15a-96b2-4f18-b5e1-351fa48db162 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:13.965503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6a15a-96b2-4f18-b5e1-351fa48db162 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:13.969444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:13.994924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 347/704] 10.222.0.121 () {56 vars in 1138 bytes} [Thu May 26 17:40:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:13.996333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6a15a-96b2-4f18-b5e1-351fa48db162 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:14.031661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 358/705] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:40:14.061623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbcf90-aa0a-45c6-9a96-e7b4f00f22a2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:14.151010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6a15a-96b2-4f18-b5e1-351fa48db162 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:14.203112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5fdbf-11c7-481f-83bc-814431464420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:14.203771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5fdbf-11c7-481f-83bc-814431464420 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:14.204119 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5fdbf-11c7-481f-83bc-814431464420 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:14.204826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5fdbf-11c7-481f-83bc-814431464420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:14.228080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbcf90-aa0a-45c6-9a96-e7b4f00f22a2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:14.261764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:14.266982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c3f34-12a0-40f3-a205-4ebc6a227841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:14.267895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c3f34-12a0-40f3-a205-4ebc6a227841 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:14.268292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c3f34-12a0-40f3-a205-4ebc6a227841 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:14.268565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c3f34-12a0-40f3-a205-4ebc6a227841 None admin] PATH_INFO: `/v3/services/network` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:14.269327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c3f34-12a0-40f3-a205-4ebc6a227841 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:14.272707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c3f34-12a0-40f3-a205-4ebc6a227841 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:14.281324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7a6c3f34-12a0-40f3-a205-4ebc6a227841 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:14.286632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:14.287729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:14.288669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 359/706] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:14 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:14.289482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20ad4-4669-4202-8784-f6af60d78b8e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:14.290095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20ad4-4669-4202-8784-f6af60d78b8e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:14.290095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20ad4-4669-4202-8784-f6af60d78b8e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:14.292066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 360/707] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:14.296423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20ad4-4669-4202-8784-f6af60d78b8e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:14.296944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5fdbf-11c7-481f-83bc-814431464420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:14.299036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5fdbf-11c7-481f-83bc-814431464420 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:14.420430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20ad4-4669-4202-8784-f6af60d78b8e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:14.437276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 348/708] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:13 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:40:14.439666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70a6ff-0a29-440b-87c5-82333cbf1b94 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:14.439954 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70a6ff-0a29-440b-87c5-82333cbf1b94 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:14.440227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70a6ff-0a29-440b-87c5-82333cbf1b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:14.464737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-eb70a6ff-0a29-440b-87c5-82333cbf1b94 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:14.464930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b093e09-74ab-4584-a535-5bf1763d91c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:14.465506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b093e09-74ab-4584-a535-5bf1763d91c3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:14.465786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b093e09-74ab-4584-a535-5bf1763d91c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:14.466061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b093e09-74ab-4584-a535-5bf1763d91c3 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:14.466898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b093e09-74ab-4584-a535-5bf1763d91c3 None admin] RBAC: Adding query filter params (name=network) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:14.467346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b093e09-74ab-4584-a535-5bf1763d91c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:14.467560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-eb70a6ff-0a29-440b-87c5-82333cbf1b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:14.468707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b093e09-74ab-4584-a535-5bf1763d91c3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:14.480339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 361/709] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:14 2022] GET /identity/v3/services?name=network => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:14.480339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b093e09-74ab-4584-a535-5bf1763d91c3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:14.557671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 349/710] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:40:14.558470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70a6ff-0a29-440b-87c5-82333cbf1b94 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:14.561485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:14.585269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:14.617684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b093e09-74ab-4584-a535-5bf1763d91c3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:14.660235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9589e8-124e-4758-bd3d-24eeb646af4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:14.661195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9589e8-124e-4758-bd3d-24eeb646af4d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:14.661574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9589e8-124e-4758-bd3d-24eeb646af4d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:14.661946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9589e8-124e-4758-bd3d-24eeb646af4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:14.675708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9589e8-124e-4758-bd3d-24eeb646af4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:14.677331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9589e8-124e-4758-bd3d-24eeb646af4d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:14.686965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70a6ff-0a29-440b-87c5-82333cbf1b94 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-ee9589e8-124e-4758-bd3d-24eeb646af4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:14.696083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:14.697625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:14.699431 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:14.699431 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:14.699431 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:14.699431 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:14.700287 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 362/711] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:14.701227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9589e8-124e-4758-bd3d-24eeb646af4d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:14.742990 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd165b1-c64f-40ff-9316-6a3dcf58f70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:14.744040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd165b1-c64f-40ff-9316-6a3dcf58f70e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:14.744558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd165b1-c64f-40ff-9316-6a3dcf58f70e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:14.745034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd165b1-c64f-40ff-9316-6a3dcf58f70e None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:14.745949 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd165b1-c64f-40ff-9316-6a3dcf58f70e None admin] RBAC: Adding query filter params (type=network) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:14.746454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd165b1-c64f-40ff-9316-6a3dcf58f70e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:14.748051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd165b1-c64f-40ff-9316-6a3dcf58f70e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:14.763071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 350/712] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:14 2022] GET /identity/v3/services?type=network => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:14.764413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd165b1-c64f-40ff-9316-6a3dcf58f70e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:14.866852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9589e8-124e-4758-bd3d-24eeb646af4d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:14.880226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:14.905741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02098665-76d6-4256-b816-87c082c47df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:14.907100 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02098665-76d6-4256-b816-87c082c47df0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:14.908242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02098665-76d6-4256-b816-87c082c47df0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:14.908985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02098665-76d6-4256-b816-87c082c47df0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:14.909973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02098665-76d6-4256-b816-87c082c47df0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:14.911661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02098665-76d6-4256-b816-87c082c47df0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:14.922676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-02098665-76d6-4256-b816-87c082c47df0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:14.938461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:14.939514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:40:14.940436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:14.943571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 363/713] 10.222.0.121 () {56 vars in 1134 bytes} [Thu May 26 17:40:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:14.950281 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02098665-76d6-4256-b816-87c082c47df0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:14.972073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd165b1-c64f-40ff-9316-6a3dcf58f70e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:15.050428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462d729-20b3-4639-8eb8-95a1a2ba0cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:15.051514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462d729-20b3-4639-8eb8-95a1a2ba0cd0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:15.052023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462d729-20b3-4639-8eb8-95a1a2ba0cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:15.052607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462d729-20b3-4639-8eb8-95a1a2ba0cd0 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:15.056641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462d729-20b3-4639-8eb8-95a1a2ba0cd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:15.058046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462d729-20b3-4639-8eb8-95a1a2ba0cd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:15.060249 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462d729-20b3-4639-8eb8-95a1a2ba0cd0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:15.092662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 351/714] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:15.099254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462d729-20b3-4639-8eb8-95a1a2ba0cd0 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:15.113733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02098665-76d6-4256-b816-87c082c47df0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:15.165720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4914d97-1f3c-4f11-aad1-a3512bca40d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:15.167373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4914d97-1f3c-4f11-aad1-a3512bca40d1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:15.168142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4914d97-1f3c-4f11-aad1-a3512bca40d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:15.168773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4914d97-1f3c-4f11-aad1-a3512bca40d1 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:15.169905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4914d97-1f3c-4f11-aad1-a3512bca40d1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:15.170779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4914d97-1f3c-4f11-aad1-a3512bca40d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:15.173121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4914d97-1f3c-4f11-aad1-a3512bca40d1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:15.180457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 364/715] 10.222.0.121 () {56 vars in 1129 bytes} [Thu May 26 17:40:14 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:15.186367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4914d97-1f3c-4f11-aad1-a3512bca40d1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:15.244286 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:15.264362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462d729-20b3-4639-8eb8-95a1a2ba0cd0 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:15.309491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4914d97-1f3c-4f11-aad1-a3512bca40d1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:15.315859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9b9cb-b44d-4531-b4ae-2e45e6c79730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:15.316680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9b9cb-b44d-4531-b4ae-2e45e6c79730 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:15.316958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9b9cb-b44d-4531-b4ae-2e45e6c79730 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:15.317232 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9b9cb-b44d-4531-b4ae-2e45e6c79730 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:15.326354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9b9cb-b44d-4531-b4ae-2e45e6c79730 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:15.328183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9b9cb-b44d-4531-b4ae-2e45e6c79730 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-69f9b9cb-b44d-4531-b4ae-2e45e6c79730 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:15.348717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:15.349705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:15.350590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:15.350590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:15.350590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:15.350590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:15.350590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:15.350590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:15.350590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:15.350590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:15.350590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 352/716] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:15.354842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226432c-68bd-4ce1-b7f1-18c7e8bfbfa6 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:15.354842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226432c-68bd-4ce1-b7f1-18c7e8bfbfa6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:15.354842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226432c-68bd-4ce1-b7f1-18c7e8bfbfa6 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:15.354842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 353/717] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:15.357686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226432c-68bd-4ce1-b7f1-18c7e8bfbfa6 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:15.359551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5980a-1772-4959-badc-ec55746dbde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:15.360242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5980a-1772-4959-badc-ec55746dbde9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:15.361145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5980a-1772-4959-badc-ec55746dbde9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:15.361443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5980a-1772-4959-badc-ec55746dbde9 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:15.362280 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5980a-1772-4959-badc-ec55746dbde9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:15.362596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5980a-1772-4959-badc-ec55746dbde9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:15.365648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5980a-1772-4959-badc-ec55746dbde9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:15.372443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:15.378860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 365/718] 10.222.0.121 () {56 vars in 1129 bytes} [Thu May 26 17:40:15 2022] GET /identity/v3/services?type=block-storage => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:15.378860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4ce24-1545-4ae5-8d99-3d2db797ae99 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:15.378860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4ce24-1545-4ae5-8d99-3d2db797ae99 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:15.378860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4ce24-1545-4ae5-8d99-3d2db797ae99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:15.412618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-42b4ce24-1545-4ae5-8d99-3d2db797ae99 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:15.413886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-42b4ce24-1545-4ae5-8d99-3d2db797ae99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:15.539472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 366/719] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:40:15.544392 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226432c-68bd-4ce1-b7f1-18c7e8bfbfa6 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:15.586449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4ce24-1545-4ae5-8d99-3d2db797ae99 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:15.599487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab91c6-1f78-42e4-9da7-8b02c6f615d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:15.600065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab91c6-1f78-42e4-9da7-8b02c6f615d5 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:15.600386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab91c6-1f78-42e4-9da7-8b02c6f615d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:15.600665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab91c6-1f78-42e4-9da7-8b02c6f615d5 None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:15.601519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab91c6-1f78-42e4-9da7-8b02c6f615d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:15.601834 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab91c6-1f78-42e4-9da7-8b02c6f615d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:15.603310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab91c6-1f78-42e4-9da7-8b02c6f615d5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:15.625288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 354/720] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:15.634847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab91c6-1f78-42e4-9da7-8b02c6f615d5 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:15.635916 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:15.714546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:15.725862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4ce24-1545-4ae5-8d99-3d2db797ae99 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:15.752579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab91c6-1f78-42e4-9da7-8b02c6f615d5 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:15.768654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592fa20-08a0-49ff-9793-db1d94625349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:15.770421 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592fa20-08a0-49ff-9793-db1d94625349 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:15.771073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592fa20-08a0-49ff-9793-db1d94625349 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:15.771594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592fa20-08a0-49ff-9793-db1d94625349 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:15.787631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592fa20-08a0-49ff-9793-db1d94625349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:15.790056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4117e-b962-4ce7-9d1a-52576d78d326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:15.790729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4117e-b962-4ce7-9d1a-52576d78d326 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:15.791339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4117e-b962-4ce7-9d1a-52576d78d326 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:15.791615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4117e-b962-4ce7-9d1a-52576d78d326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:15.792516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4117e-b962-4ce7-9d1a-52576d78d326 None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=c0889a69a018452d8de1924072739d17) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:15.792830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4117e-b962-4ce7-9d1a-52576d78d326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:15.794268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4117e-b962-4ce7-9d1a-52576d78d326 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7592fa20-08a0-49ff-9793-db1d94625349 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:15.798499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:15.799628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:15.800448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:15.800448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:15.800448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:15.800448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:15.800448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:15.800448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:15.800448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:15.800448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:15.800448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:15.800448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:15.800448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:15.800448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:15.803479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 367/722] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:40:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:15.803605 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 355/722] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:15 2022] GET /identity/v3/role_assignments?user.id=c0889a69a018452d8de1924072739d17&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=4f574961798246a4b12ad2276e67685d => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:15.813547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4117e-b962-4ce7-9d1a-52576d78d326 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:15.981161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4117e-b962-4ce7-9d1a-52576d78d326 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:16.030252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:16.037547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e3ff0-3acd-4d6a-b888-c4f2074fa0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:16.042622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e3ff0-3acd-4d6a-b888-c4f2074fa0cf None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:16.042622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e3ff0-3acd-4d6a-b888-c4f2074fa0cf None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:16.042622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e3ff0-3acd-4d6a-b888-c4f2074fa0cf None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:16.042622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e3ff0-3acd-4d6a-b888-c4f2074fa0cf None admin] RBAC: Adding query filter params (name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:16.042622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e3ff0-3acd-4d6a-b888-c4f2074fa0cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:16.042622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e3ff0-3acd-4d6a-b888-c4f2074fa0cf None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:16.051903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 356/723] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:40:15 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:16.057421 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592fa20-08a0-49ff-9793-db1d94625349 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:16.205556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:16.213143 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592fa20-08a0-49ff-9793-db1d94625349 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:16.252300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986f46a-a0df-4536-9871-49f7e430d830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:16.253379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986f46a-a0df-4536-9871-49f7e430d830 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:16.253661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986f46a-a0df-4536-9871-49f7e430d830 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:16.253932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986f46a-a0df-4536-9871-49f7e430d830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:16.265435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986f46a-a0df-4536-9871-49f7e430d830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:16.267726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986f46a-a0df-4536-9871-49f7e430d830 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-c986f46a-a0df-4536-9871-49f7e430d830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:16.294872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:16.296076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:16.297175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:16.297175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:16.297175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:16.297175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:16.297175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 368/724] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:16.303060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e3ff0-3acd-4d6a-b888-c4f2074fa0cf None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:16.416905 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e3ff0-3acd-4d6a-b888-c4f2074fa0cf None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:16.452991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9d724-08f8-4451-888a-2fa54da7a339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:16.453671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9d724-08f8-4451-888a-2fa54da7a339 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:16.453986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9d724-08f8-4451-888a-2fa54da7a339 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:16.454448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9d724-08f8-4451-888a-2fa54da7a339 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:16.455599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9d724-08f8-4451-888a-2fa54da7a339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:16.455909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9d724-08f8-4451-888a-2fa54da7a339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:16.457372 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9d724-08f8-4451-888a-2fa54da7a339 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:16.475194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 357/725] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:16.476693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986f46a-a0df-4536-9871-49f7e430d830 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:16.547705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a613d-7f56-4dd7-a3e4-bd144f9af790 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:16.548110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a613d-7f56-4dd7-a3e4-bd144f9af790 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:16.548361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a613d-7f56-4dd7-a3e4-bd144f9af790 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:16.549740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 358/726] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:16.555941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067874d-cf4e-4333-943e-ccd5afea6954 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:16.556303 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067874d-cf4e-4333-943e-ccd5afea6954 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:16.556732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067874d-cf4e-4333-943e-ccd5afea6954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:16.585774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b067874d-cf4e-4333-943e-ccd5afea6954 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:16.586987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b067874d-cf4e-4333-943e-ccd5afea6954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:16.635508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986f46a-a0df-4536-9871-49f7e430d830 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:16.685506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:16.699943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb9b67-5baa-4cf8-8c2e-3651b88834d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:16.701164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb9b67-5baa-4cf8-8c2e-3651b88834d9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:16.701588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb9b67-5baa-4cf8-8c2e-3651b88834d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:16.702027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb9b67-5baa-4cf8-8c2e-3651b88834d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:16.710322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb9b67-5baa-4cf8-8c2e-3651b88834d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:16.712033 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb9b67-5baa-4cf8-8c2e-3651b88834d9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:16.725366 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 359/727] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-50fb9b67-5baa-4cf8-8c2e-3651b88834d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:16.730362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:16.741289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:16.742257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:16.742257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:16.742257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:16.742257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:16.742257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:16.742257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:16.742257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:16.742257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:16.742257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:40:16.742257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:16.742257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 369/728] 10.222.0.121 () {56 vars in 1122 bytes} [Thu May 26 17:40:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:16.742853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067874d-cf4e-4333-943e-ccd5afea6954 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:16.759016 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb9b67-5baa-4cf8-8c2e-3651b88834d9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:16.770338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:16.885377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb9b67-5baa-4cf8-8c2e-3651b88834d9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:16.925737 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104c209-b35b-46cd-9d13-ac4ef61d68d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:16.927944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104c209-b35b-46cd-9d13-ac4ef61d68d0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:16.928533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104c209-b35b-46cd-9d13-ac4ef61d68d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:16.929264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104c209-b35b-46cd-9d13-ac4ef61d68d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:16.942166 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067874d-cf4e-4333-943e-ccd5afea6954 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:16.943341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104c209-b35b-46cd-9d13-ac4ef61d68d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-b104c209-b35b-46cd-9d13-ac4ef61d68d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:16.951539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:16.952479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:16.953317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:16.953317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:16.953317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:16.953317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:16.953317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:16.953317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:16.953317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:16.953317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:16.953317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:16.953317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:16.953317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:40:16.953317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:16.957750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 370/729] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:40:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:16.959101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104c209-b35b-46cd-9d13-ac4ef61d68d0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:17.011570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcafe97-b295-4713-9c72-0d66368eeae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:17.012277 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcafe97-b295-4713-9c72-0d66368eeae0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:17.012550 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcafe97-b295-4713-9c72-0d66368eeae0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:17.012831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcafe97-b295-4713-9c72-0d66368eeae0 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:17.013748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcafe97-b295-4713-9c72-0d66368eeae0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:17.014061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcafe97-b295-4713-9c72-0d66368eeae0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:17.015578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcafe97-b295-4713-9c72-0d66368eeae0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:17.034071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:17.044159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 360/730] 10.222.0.121 () {56 vars in 1154 bytes} [Thu May 26 17:40:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:17.049771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcafe97-b295-4713-9c72-0d66368eeae0 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:17.074187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104c209-b35b-46cd-9d13-ac4ef61d68d0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:17.104440 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:17.119757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19343e7-38aa-4dc7-adb1-7e306d81a03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:17.120641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19343e7-38aa-4dc7-adb1-7e306d81a03c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:17.120919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19343e7-38aa-4dc7-adb1-7e306d81a03c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:17.121192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19343e7-38aa-4dc7-adb1-7e306d81a03c None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:17.121943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19343e7-38aa-4dc7-adb1-7e306d81a03c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:17.122244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19343e7-38aa-4dc7-adb1-7e306d81a03c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:17.123839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19343e7-38aa-4dc7-adb1-7e306d81a03c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:17.139149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 371/731] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:40:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:17.141127 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19343e7-38aa-4dc7-adb1-7e306d81a03c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:17.260944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19343e7-38aa-4dc7-adb1-7e306d81a03c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:17.291248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:17.303547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8dd6c8-f234-49fd-95fc-43e97124acad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:17.304346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8dd6c8-f234-49fd-95fc-43e97124acad None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:17.304623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8dd6c8-f234-49fd-95fc-43e97124acad None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:17.304894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8dd6c8-f234-49fd-95fc-43e97124acad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:17.305064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcafe97-b295-4713-9c72-0d66368eeae0 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:17.319471 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8dd6c8-f234-49fd-95fc-43e97124acad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:17.320989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8dd6c8-f234-49fd-95fc-43e97124acad None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-9c8dd6c8-f234-49fd-95fc-43e97124acad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:17.341186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:17.343271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:17.345995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:17.345995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:17.345995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:17.345995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:17.347180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 372/732] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:17.349516 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8dd6c8-f234-49fd-95fc-43e97124acad None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:17.447739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd496ad2-b269-4109-a37e-88d48d4e42c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:17.448344 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd496ad2-b269-4109-a37e-88d48d4e42c8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:17.448635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd496ad2-b269-4109-a37e-88d48d4e42c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:17.448908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd496ad2-b269-4109-a37e-88d48d4e42c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:17.466852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8dd6c8-f234-49fd-95fc-43e97124acad None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:17.510565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9b7f7-986b-479b-be9b-c795a901260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:17.511307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9b7f7-986b-479b-be9b-c795a901260d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:17.511586 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9b7f7-986b-479b-be9b-c795a901260d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:17.511859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9b7f7-986b-479b-be9b-c795a901260d None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:17.512597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9b7f7-986b-479b-be9b-c795a901260d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:17.512903 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9b7f7-986b-479b-be9b-c795a901260d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:17.514311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9b7f7-986b-479b-be9b-c795a901260d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:17.530734 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 373/733] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:17.538767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9b7f7-986b-479b-be9b-c795a901260d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:17.615570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:17.619299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd496ad2-b269-4109-a37e-88d48d4e42c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:17.620967 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd496ad2-b269-4109-a37e-88d48d4e42c8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:17.679963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9b7f7-986b-479b-be9b-c795a901260d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:17.699711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:17.718729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf32022-a78d-490b-9c55-02422a562093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:17.719575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf32022-a78d-490b-9c55-02422a562093 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:17.720173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf32022-a78d-490b-9c55-02422a562093 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:17.720534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf32022-a78d-490b-9c55-02422a562093 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:17.729331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf32022-a78d-490b-9c55-02422a562093 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:17.730937 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf32022-a78d-490b-9c55-02422a562093 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-1cf32022-a78d-490b-9c55-02422a562093 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:17.747862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:17.750045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 374/734] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:40:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f799f2f-cb75-4a2a-9dc7-751fb6de2f87 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:17.753009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f799f2f-cb75-4a2a-9dc7-751fb6de2f87 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:17.754442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f799f2f-cb75-4a2a-9dc7-751fb6de2f87 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:17.754442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 375/735] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:17 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:17.759067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb589b-ac2e-4bc1-acf8-ae913122b358 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:17.759355 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb589b-ac2e-4bc1-acf8-ae913122b358 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:17.759630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb589b-ac2e-4bc1-acf8-ae913122b358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:17.773313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 361/736] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:17 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:40:17.774276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd496ad2-b269-4109-a37e-88d48d4e42c8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:17.778057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b1fb589b-ac2e-4bc1-acf8-ae913122b358 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:17.779422 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b1fb589b-ac2e-4bc1-acf8-ae913122b358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:17.859904 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 376/737] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:17 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:40:17.860970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb589b-ac2e-4bc1-acf8-ae913122b358 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:17.900614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:17.914220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd496ad2-b269-4109-a37e-88d48d4e42c8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:17.952170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d1fde-da7a-4f5b-8920-28d3a5b0f24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:17.952863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d1fde-da7a-4f5b-8920-28d3a5b0f24f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:17.953212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d1fde-da7a-4f5b-8920-28d3a5b0f24f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:17.953484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d1fde-da7a-4f5b-8920-28d3a5b0f24f None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:17.955297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d1fde-da7a-4f5b-8920-28d3a5b0f24f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:17.955609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d1fde-da7a-4f5b-8920-28d3a5b0f24f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:17.957042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d1fde-da7a-4f5b-8920-28d3a5b0f24f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:17.975295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:17.983439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 362/738] 10.222.0.121 () {56 vars in 1140 bytes} [Thu May 26 17:40:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:17.986713 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d1fde-da7a-4f5b-8920-28d3a5b0f24f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:17.995622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb589b-ac2e-4bc1-acf8-ae913122b358 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:18.046733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5b019-2fb6-4233-868f-8e13694b9de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:18.049122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5b019-2fb6-4233-868f-8e13694b9de3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:18.049612 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5b019-2fb6-4233-868f-8e13694b9de3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:18.050322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5b019-2fb6-4233-868f-8e13694b9de3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:18.063290 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5b019-2fb6-4233-868f-8e13694b9de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:18.065186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5b019-2fb6-4233-868f-8e13694b9de3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-09e5b019-2fb6-4233-868f-8e13694b9de3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:18.084965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:18.086365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:18.087284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:18.087284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:18.087284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:18.087284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:18.087284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 377/739] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:18.087559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5b019-2fb6-4233-868f-8e13694b9de3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:18.114643 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d1fde-da7a-4f5b-8920-28d3a5b0f24f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:18.153275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143792b9-d498-4d4c-82d5-25fbec2d219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:18.154098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143792b9-d498-4d4c-82d5-25fbec2d219c None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:18.154485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143792b9-d498-4d4c-82d5-25fbec2d219c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:18.154974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143792b9-d498-4d4c-82d5-25fbec2d219c None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:18.155917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143792b9-d498-4d4c-82d5-25fbec2d219c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:18.158363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143792b9-d498-4d4c-82d5-25fbec2d219c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:18.202892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:18.223095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5b019-2fb6-4233-868f-8e13694b9de3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:18.258437 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d52f7d-f2fb-49b6-8cbc-6521031c6911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:18.259113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d52f7d-f2fb-49b6-8cbc-6521031c6911 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:18.259392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d52f7d-f2fb-49b6-8cbc-6521031c6911 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:18.259762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d52f7d-f2fb-49b6-8cbc-6521031c6911 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:18.275999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 363/740] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:40:17 2022] POST /identity/v3/services => generated 241 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:40:18.277452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143792b9-d498-4d4c-82d5-25fbec2d219c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:18.379418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:18.408881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d52f7d-f2fb-49b6-8cbc-6521031c6911 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:18.412401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d52f7d-f2fb-49b6-8cbc-6521031c6911 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:18.446470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143792b9-d498-4d4c-82d5-25fbec2d219c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:18.489540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebbe96-3166-4519-92fb-9a3dd9c69cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:18.490284 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebbe96-3166-4519-92fb-9a3dd9c69cfe None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:18.491944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebbe96-3166-4519-92fb-9a3dd9c69cfe None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:18.491944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebbe96-3166-4519-92fb-9a3dd9c69cfe None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:18.491944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebbe96-3166-4519-92fb-9a3dd9c69cfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:18.492467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebbe96-3166-4519-92fb-9a3dd9c69cfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:18.494440 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebbe96-3166-4519-92fb-9a3dd9c69cfe None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:18.508162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 364/741] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:18.509728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a75e3-8cf0-473e-a451-26887d3704c9 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:18.510020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a75e3-8cf0-473e-a451-26887d3704c9 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:18.510291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a75e3-8cf0-473e-a451-26887d3704c9 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:18.512369 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 365/742] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:18.513070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a75e3-8cf0-473e-a451-26887d3704c9 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:18.525023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:18.553601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 378/743] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:18 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:40:18.553601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2681f-04d2-4a3e-9783-9b5f53525c3c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:18.553601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2681f-04d2-4a3e-9783-9b5f53525c3c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:18.553601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2681f-04d2-4a3e-9783-9b5f53525c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:18.570112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-68a2681f-04d2-4a3e-9783-9b5f53525c3c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:18.571347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-68a2681f-04d2-4a3e-9783-9b5f53525c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:18.643405 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a75e3-8cf0-473e-a451-26887d3704c9 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:18.647020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 379/744] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:18 2022] POST /identity/v3/auth/tokens => generated 899 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:40:18.647020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2681f-04d2-4a3e-9783-9b5f53525c3c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:18.664433 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:18.685115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd9c91-0df2-4f29-83ae-73109b0ce104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:18.685913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd9c91-0df2-4f29-83ae-73109b0ce104 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:18.686190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd9c91-0df2-4f29-83ae-73109b0ce104 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:18.686459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd9c91-0df2-4f29-83ae-73109b0ce104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:18.694489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd9c91-0df2-4f29-83ae-73109b0ce104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:18.695952 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd9c91-0df2-4f29-83ae-73109b0ce104 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-ecdd9c91-0df2-4f29-83ae-73109b0ce104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:18.709802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:18.710722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:18.711677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:18.711677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:18.711677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:18.711677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:18.711677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:18.711677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:18.711677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:18.711677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:18.711677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 366/745] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:18.712620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212b69d-3803-434d-8645-cb6be723180a None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:18.712620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212b69d-3803-434d-8645-cb6be723180a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:18.712913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212b69d-3803-434d-8645-cb6be723180a None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:18.718280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 367/746] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:18.718280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212b69d-3803-434d-8645-cb6be723180a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:18.762969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2681f-04d2-4a3e-9783-9b5f53525c3c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:18.777854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:18.801825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916aa51-3a6c-46ff-8662-fe4e1afe2c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:18.803744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916aa51-3a6c-46ff-8662-fe4e1afe2c88 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:18.804175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916aa51-3a6c-46ff-8662-fe4e1afe2c88 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:18.804595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916aa51-3a6c-46ff-8662-fe4e1afe2c88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:18.821351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916aa51-3a6c-46ff-8662-fe4e1afe2c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:18.823551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916aa51-3a6c-46ff-8662-fe4e1afe2c88 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:18.831099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212b69d-3803-434d-8645-cb6be723180a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-c916aa51-3a6c-46ff-8662-fe4e1afe2c88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:18.843222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:18.844325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:18.845387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:18.845387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:18.845387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:18.845387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:18.845387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 380/747] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:18.845387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916aa51-3a6c-46ff-8662-fe4e1afe2c88 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:18.874656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e4523-710b-47d6-9b7b-8a43a279b334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:18.875706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e4523-710b-47d6-9b7b-8a43a279b334 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:18.876629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e4523-710b-47d6-9b7b-8a43a279b334 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:18.877088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e4523-710b-47d6-9b7b-8a43a279b334 None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:18.877988 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e4523-710b-47d6-9b7b-8a43a279b334 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:18.880163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e4523-710b-47d6-9b7b-8a43a279b334 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:18.953085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916aa51-3a6c-46ff-8662-fe4e1afe2c88 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:18.961828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 368/748] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:40:18 2022] POST /identity/v3/services => generated 252 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:40:18.961828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9e9e6-6e12-4726-9e2d-9739574eecc5 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:18.961828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9e9e6-6e12-4726-9e2d-9739574eecc5 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:18.961828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9e9e6-6e12-4726-9e2d-9739574eecc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:18.978751 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c2a9e9e6-6e12-4726-9e2d-9739574eecc5 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:18.980248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c2a9e9e6-6e12-4726-9e2d-9739574eecc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:19.009677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ee04b-b82e-4c0e-b281-fa38d9eb05b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:19.010374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ee04b-b82e-4c0e-b281-fa38d9eb05b5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:19.010649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ee04b-b82e-4c0e-b281-fa38d9eb05b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:19.010945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ee04b-b82e-4c0e-b281-fa38d9eb05b5 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:19.011857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ee04b-b82e-4c0e-b281-fa38d9eb05b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:19.012157 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ee04b-b82e-4c0e-b281-fa38d9eb05b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:19.013646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ee04b-b82e-4c0e-b281-fa38d9eb05b5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:19.022192 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:19.031125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 381/749] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:19.032642 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ee04b-b82e-4c0e-b281-fa38d9eb05b5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:19.066480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:19.084640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 369/750] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:18 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:19.085839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9e9e6-6e12-4726-9e2d-9739574eecc5 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:19.165450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ee04b-b82e-4c0e-b281-fa38d9eb05b5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:19.207006 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9e9e6-6e12-4726-9e2d-9739574eecc5 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:19.209553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29bc05-4029-4827-8c5a-a9fc13932370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:19.210235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29bc05-4029-4827-8c5a-a9fc13932370 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:19.210509 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29bc05-4029-4827-8c5a-a9fc13932370 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:19.210779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29bc05-4029-4827-8c5a-a9fc13932370 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:19.211539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29bc05-4029-4827-8c5a-a9fc13932370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:19.211841 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29bc05-4029-4827-8c5a-a9fc13932370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:19.213219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29bc05-4029-4827-8c5a-a9fc13932370 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:19.227963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 382/751] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:19.229819 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29bc05-4029-4827-8c5a-a9fc13932370 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:19.264827 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217dfda-8743-463b-b2e5-454150ad6e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:19.265436 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217dfda-8743-463b-b2e5-454150ad6e19 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:19.265700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217dfda-8743-463b-b2e5-454150ad6e19 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:19.265966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217dfda-8743-463b-b2e5-454150ad6e19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:19.267553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217dfda-8743-463b-b2e5-454150ad6e19 None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=485c784a043541619ffffc657b21ab97) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:19.268126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217dfda-8743-463b-b2e5-454150ad6e19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:19.269623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217dfda-8743-463b-b2e5-454150ad6e19 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:19.277177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 370/752] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:19 2022] GET /identity/v3/role_assignments?user.id=485c784a043541619ffffc657b21ab97&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:19.278233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217dfda-8743-463b-b2e5-454150ad6e19 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:19.328300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:19.342019 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:19.369136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29bc05-4029-4827-8c5a-a9fc13932370 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:19.413834 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217dfda-8743-463b-b2e5-454150ad6e19 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:19.416266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca617e-735e-494b-84e4-9e639d5f94a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:19.417339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca617e-735e-494b-84e4-9e639d5f94a9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:19.417625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca617e-735e-494b-84e4-9e639d5f94a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:19.417899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca617e-735e-494b-84e4-9e639d5f94a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:19.436672 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca617e-735e-494b-84e4-9e639d5f94a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-a5ca617e-735e-494b-84e4-9e639d5f94a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:19.447729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:19.448730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:19.449569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 383/753] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:40:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:19.456096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca617e-735e-494b-84e4-9e639d5f94a9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:19.461357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df84ff-1de7-412b-b060-6afe6e69c7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:19.462571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df84ff-1de7-412b-b060-6afe6e69c7ca None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:19.463278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df84ff-1de7-412b-b060-6afe6e69c7ca None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:19.463897 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df84ff-1de7-412b-b060-6afe6e69c7ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:19.473478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df84ff-1de7-412b-b060-6afe6e69c7ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:19.476265 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df84ff-1de7-412b-b060-6afe6e69c7ca None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-68df84ff-1de7-412b-b060-6afe6e69c7ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:19.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:19.497454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:19.499016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:19.499016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:19.499016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:19.499016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:19.499016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:19.499016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:19.499016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:19.499016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:19.502310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 371/754] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:19.508084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df84ff-1de7-412b-b060-6afe6e69c7ca None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:19.668101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:19.683795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca617e-735e-494b-84e4-9e639d5f94a9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:19.688096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df84ff-1de7-412b-b060-6afe6e69c7ca None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:19.745208 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b2e2c-dd62-4717-aa1a-a96fc9338f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:19.745876 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b2e2c-dd62-4717-aa1a-a96fc9338f63 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:19.746152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b2e2c-dd62-4717-aa1a-a96fc9338f63 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:19.746419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b2e2c-dd62-4717-aa1a-a96fc9338f63 None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:19.747238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b2e2c-dd62-4717-aa1a-a96fc9338f63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:19.747542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b2e2c-dd62-4717-aa1a-a96fc9338f63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:19.749116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b2e2c-dd62-4717-aa1a-a96fc9338f63 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:19.760101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be916370-2eea-4cab-a1d1-c42eb021ccb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:19.760790 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be916370-2eea-4cab-a1d1-c42eb021ccb9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:19.761071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be916370-2eea-4cab-a1d1-c42eb021ccb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:19.761350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be916370-2eea-4cab-a1d1-c42eb021ccb9 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:19.762102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be916370-2eea-4cab-a1d1-c42eb021ccb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:19.762405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be916370-2eea-4cab-a1d1-c42eb021ccb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:19.763020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 372/755] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:19.763816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be916370-2eea-4cab-a1d1-c42eb021ccb9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:19.769733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b2e2c-dd62-4717-aa1a-a96fc9338f63 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:19.783496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 384/756] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:40:19 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:19.789265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be916370-2eea-4cab-a1d1-c42eb021ccb9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:19.795426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:19.931390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be916370-2eea-4cab-a1d1-c42eb021ccb9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:19.970977 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72567877-92e1-4ee2-9f84-e2e42509e5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:19.972341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72567877-92e1-4ee2-9f84-e2e42509e5e7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:19.972821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72567877-92e1-4ee2-9f84-e2e42509e5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:19.973289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72567877-92e1-4ee2-9f84-e2e42509e5e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:19.985757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72567877-92e1-4ee2-9f84-e2e42509e5e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:19.988244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72567877-92e1-4ee2-9f84-e2e42509e5e7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:20.005722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b2e2c-dd62-4717-aa1a-a96fc9338f63 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-72567877-92e1-4ee2-9f84-e2e42509e5e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:20.007804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:20.008658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:20.009532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:20.009532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:20.009532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:20.009532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:20.011843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 385/757] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:20.017367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72567877-92e1-4ee2-9f84-e2e42509e5e7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:20.081944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2502c9-3f0a-4798-9ded-ef36af95ade4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:20.085031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2502c9-3f0a-4798-9ded-ef36af95ade4 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:20.085519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2502c9-3f0a-4798-9ded-ef36af95ade4 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:20.085984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2502c9-3f0a-4798-9ded-ef36af95ade4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:20.087069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2502c9-3f0a-4798-9ded-ef36af95ade4 None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=eb9ba9c73956403ba870f6f6b0e687aa) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:20.088481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2502c9-3f0a-4798-9ded-ef36af95ade4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:20.095271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2502c9-3f0a-4798-9ded-ef36af95ade4 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:20.106415 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 373/758] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:19 2022] GET /identity/v3/role_assignments?user.id=eb9ba9c73956403ba870f6f6b0e687aa&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=4f574961798246a4b12ad2276e67685d => generated 260 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:20.121789 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:20.179521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72567877-92e1-4ee2-9f84-e2e42509e5e7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:20.217799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e54a8c-ee08-4126-a70a-519e519c7ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:20.218811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e54a8c-ee08-4126-a70a-519e519c7ca1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:20.219418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e54a8c-ee08-4126-a70a-519e519c7ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:20.220632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e54a8c-ee08-4126-a70a-519e519c7ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:20.221589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e54a8c-ee08-4126-a70a-519e519c7ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:20.222092 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e54a8c-ee08-4126-a70a-519e519c7ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:20.223686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e54a8c-ee08-4126-a70a-519e519c7ca1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:20.239123 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 386/759] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:20.247960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2502c9-3f0a-4798-9ded-ef36af95ade4 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:20.274864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:20.386086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2502c9-3f0a-4798-9ded-ef36af95ade4 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:20.425333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfe3ef-d842-40bb-bae1-f62870837d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:20.426511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfe3ef-d842-40bb-bae1-f62870837d11 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:20.427048 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfe3ef-d842-40bb-bae1-f62870837d11 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:20.427517 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfe3ef-d842-40bb-bae1-f62870837d11 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:20.435813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cfe3ef-d842-40bb-bae1-f62870837d11 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:20.437537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cfe3ef-d842-40bb-bae1-f62870837d11 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-a7cfe3ef-d842-40bb-bae1-f62870837d11 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:20.453721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:20.455960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:20.456892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:20.456892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:20.456892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:20.456892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:20.456892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:20.456892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:20.456892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:20.456892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:20.456892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:40:20.456892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:20.459694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 374/760] 10.222.0.121 () {56 vars in 1098 bytes} [Thu May 26 17:40:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:20.468282 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e54a8c-ee08-4126-a70a-519e519c7ca1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:20.606945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e54a8c-ee08-4126-a70a-519e519c7ca1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:20.627036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:20.652218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4c182-e4d4-4a0c-8fe2-fe21b81fad49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:20.652897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4c182-e4d4-4a0c-8fe2-fe21b81fad49 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:20.653178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4c182-e4d4-4a0c-8fe2-fe21b81fad49 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:20.653450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4c182-e4d4-4a0c-8fe2-fe21b81fad49 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:20.654283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4c182-e4d4-4a0c-8fe2-fe21b81fad49 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:20.654587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4c182-e4d4-4a0c-8fe2-fe21b81fad49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:20.656048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4c182-e4d4-4a0c-8fe2-fe21b81fad49 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:20.683237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 387/761] 10.222.0.121 () {56 vars in 1138 bytes} [Thu May 26 17:40:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:20.688815 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfe3ef-d842-40bb-bae1-f62870837d11 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:20.774677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:20.775308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49571ee8-30b9-434e-b2c8-6ce4bc360dee None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:20.775550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49571ee8-30b9-434e-b2c8-6ce4bc360dee None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:20.775831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49571ee8-30b9-434e-b2c8-6ce4bc360dee None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:20.778368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 388/762] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:20 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:20.783062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45b242-0ea8-4020-8659-4599c69037b2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:20.783405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45b242-0ea8-4020-8659-4599c69037b2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:20.783609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45b242-0ea8-4020-8659-4599c69037b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:20.806387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfe3ef-d842-40bb-bae1-f62870837d11 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:20.819879 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-db45b242-0ea8-4020-8659-4599c69037b2 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:20.821106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-db45b242-0ea8-4020-8659-4599c69037b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:20.853637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d93a13-916a-4f39-98d0-2c8969701f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:20.857444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d93a13-916a-4f39-98d0-2c8969701f1b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:20.857951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d93a13-916a-4f39-98d0-2c8969701f1b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:20.858441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d93a13-916a-4f39-98d0-2c8969701f1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:20.877638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d93a13-916a-4f39-98d0-2c8969701f1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:20.879622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d93a13-916a-4f39-98d0-2c8969701f1b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-b7d93a13-916a-4f39-98d0-2c8969701f1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:20.908861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:20.909806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:20.910679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:20.910679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:20.910679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:20.910679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:20.913047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 375/763] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:20.918347 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d93a13-916a-4f39-98d0-2c8969701f1b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:20.941751 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 389/764] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:20 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:20.983153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45b242-0ea8-4020-8659-4599c69037b2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:21.066931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d93a13-916a-4f39-98d0-2c8969701f1b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:21.088065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:21.103893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1f65c-763d-4f59-a77a-8f974b2a2e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:21.104565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1f65c-763d-4f59-a77a-8f974b2a2e2f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:21.104841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1f65c-763d-4f59-a77a-8f974b2a2e2f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:21.105110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1f65c-763d-4f59-a77a-8f974b2a2e2f None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:21.105834 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1f65c-763d-4f59-a77a-8f974b2a2e2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:21.106138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1f65c-763d-4f59-a77a-8f974b2a2e2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:21.107552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1f65c-763d-4f59-a77a-8f974b2a2e2f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:21.137968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 376/765] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:21.143803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1f65c-763d-4f59-a77a-8f974b2a2e2f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:21.183482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45b242-0ea8-4020-8659-4599c69037b2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:21.194874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:21.265253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9265c-e3cf-4c9d-bbeb-1074ff634e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:21.267489 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9265c-e3cf-4c9d-bbeb-1074ff634e46 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:21.267794 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9265c-e3cf-4c9d-bbeb-1074ff634e46 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:21.268041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9265c-e3cf-4c9d-bbeb-1074ff634e46 None admin] PATH_INFO: `/v3/services/network` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:21.268706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9265c-e3cf-4c9d-bbeb-1074ff634e46 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:21.270108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9265c-e3cf-4c9d-bbeb-1074ff634e46 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:21.300398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1f65c-763d-4f59-a77a-8f974b2a2e2f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f0b9265c-e3cf-4c9d-bbeb-1074ff634e46 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:21.302122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:21.303118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:40:21.304025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:21.304768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 390/766] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:20 2022] GET /identity/v3/services/network => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:21.309409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9265c-e3cf-4c9d-bbeb-1074ff634e46 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:21.356273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2318e3-2066-445c-b606-8783f5bddff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:21.356998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2318e3-2066-445c-b606-8783f5bddff8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:21.357279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2318e3-2066-445c-b606-8783f5bddff8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:21.357553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2318e3-2066-445c-b606-8783f5bddff8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:21.366783 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2318e3-2066-445c-b606-8783f5bddff8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:21.368845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2318e3-2066-445c-b606-8783f5bddff8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-7e2318e3-2066-445c-b606-8783f5bddff8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:21.387084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:21.388046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:21.388927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:21.388927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:21.388927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:21.388927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:21.388927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:21.388927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:21.388927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:21.388927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:21.388927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 377/767] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:21.406892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2318e3-2066-445c-b606-8783f5bddff8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:21.468774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9265c-e3cf-4c9d-bbeb-1074ff634e46 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:21.504234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:21.510511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962b6ba-2df0-47d5-ab64-179bdf15677a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:21.511236 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962b6ba-2df0-47d5-ab64-179bdf15677a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:21.511520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962b6ba-2df0-47d5-ab64-179bdf15677a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:21.511798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962b6ba-2df0-47d5-ab64-179bdf15677a None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:21.512513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962b6ba-2df0-47d5-ab64-179bdf15677a None admin] RBAC: Adding query filter params (name=network) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:21.512821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962b6ba-2df0-47d5-ab64-179bdf15677a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:21.514869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962b6ba-2df0-47d5-ab64-179bdf15677a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:21.520184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 391/768] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:21 2022] GET /identity/v3/services?name=network => generated 125 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:21.525251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962b6ba-2df0-47d5-ab64-179bdf15677a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:21.586347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:21.619287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2318e3-2066-445c-b606-8783f5bddff8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:21.684846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c38a2-771e-4741-a4bf-005471ac64fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:21.685980 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c38a2-771e-4741-a4bf-005471ac64fe None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:21.686460 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c38a2-771e-4741-a4bf-005471ac64fe None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:21.687026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c38a2-771e-4741-a4bf-005471ac64fe None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:21.688728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c38a2-771e-4741-a4bf-005471ac64fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:21.689338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c38a2-771e-4741-a4bf-005471ac64fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:21.691084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c38a2-771e-4741-a4bf-005471ac64fe None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:21.697978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962b6ba-2df0-47d5-ab64-179bdf15677a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:21.719302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 378/769] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:21.732809 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c38a2-771e-4741-a4bf-005471ac64fe None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:21.738485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853ab34-b839-49f7-8de7-e3b5fc114553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:21.739328 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853ab34-b839-49f7-8de7-e3b5fc114553 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:21.739611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853ab34-b839-49f7-8de7-e3b5fc114553 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:21.744865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853ab34-b839-49f7-8de7-e3b5fc114553 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:21.744865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853ab34-b839-49f7-8de7-e3b5fc114553 None admin] RBAC: Adding query filter params (type=network) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:21.744865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853ab34-b839-49f7-8de7-e3b5fc114553 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:21.744865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853ab34-b839-49f7-8de7-e3b5fc114553 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:21.748337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 392/770] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:21 2022] GET /identity/v3/services?type=network => generated 352 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:21.754276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853ab34-b839-49f7-8de7-e3b5fc114553 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:21.884882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c38a2-771e-4741-a4bf-005471ac64fe None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:21.897282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:21.928095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:21.939526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853ab34-b839-49f7-8de7-e3b5fc114553 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:21.959496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f92f2-4854-4e7c-888e-36bc7293f83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:21.960727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f92f2-4854-4e7c-888e-36bc7293f83c None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:21.961181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f92f2-4854-4e7c-888e-36bc7293f83c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:21.961639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f92f2-4854-4e7c-888e-36bc7293f83c None admin] PATH_INFO: `/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/c0889a69a018452d8de1924072739d17/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:21.964480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f92f2-4854-4e7c-888e-36bc7293f83c None admin] RBAC: Authorizing `identity:create_grant(project_id=d61dea65b34b47a186151c6a63bc7d79, user_id=c0889a69a018452d8de1924072739d17, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:21.966330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f92f2-4854-4e7c-888e-36bc7293f83c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:21.994743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cade25-f7e2-45ef-9f24-d8831819831f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:21.995807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cade25-f7e2-45ef-9f24-d8831819831f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:21.996262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cade25-f7e2-45ef-9f24-d8831819831f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:21.996729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cade25-f7e2-45ef-9f24-d8831819831f None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:21.997716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cade25-f7e2-45ef-9f24-d8831819831f None admin] RBAC: Adding query filter params (interface=public, service_id=6c7147f11b514864b93be674550fcab4, region_id=RegionOne) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:21.998261 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cade25-f7e2-45ef-9f24-d8831819831f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:22.000054 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cade25-f7e2-45ef-9f24-d8831819831f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:22.008808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 393/771] 10.222.0.121 () {56 vars in 1256 bytes} [Thu May 26 17:40:21 2022] GET /identity/v3/endpoints?service_id=6c7147f11b514864b93be674550fcab4&interface=public®ion_id=RegionOne => generated 195 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:22.010091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecf765-ad29-4632-ae0f-e250a115ccd4 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:22.010378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecf765-ad29-4632-ae0f-e250a115ccd4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:22.010655 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecf765-ad29-4632-ae0f-e250a115ccd4 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:22.012178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 394/772] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:22.015229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecf765-ad29-4632-ae0f-e250a115ccd4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:22.039534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 379/773] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:40:21 2022] PUT /identity/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/c0889a69a018452d8de1924072739d17/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:40:22.041380 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32cee8-81ba-4124-8ced-bb8192753a09 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:22.044139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32cee8-81ba-4124-8ced-bb8192753a09 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:22.044139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32cee8-81ba-4124-8ced-bb8192753a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:22.065543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1a32cee8-81ba-4124-8ced-bb8192753a09 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:22.067152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1a32cee8-81ba-4124-8ced-bb8192753a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:22.163000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 380/774] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:22 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:22.164752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23925cb-a601-4595-80ce-d1afec533b39 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:22.165226 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23925cb-a601-4595-80ce-d1afec533b39 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:22.165681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23925cb-a601-4595-80ce-d1afec533b39 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:22.172104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 381/775] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:22 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:22.177106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6db4b3-0121-4116-9bf3-6cb50e40992f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:22.178459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6db4b3-0121-4116-9bf3-6cb50e40992f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:22.179127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6db4b3-0121-4116-9bf3-6cb50e40992f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:22.189841 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecf765-ad29-4632-ae0f-e250a115ccd4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:22.204469 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-4c6db4b3-0121-4116-9bf3-6cb50e40992f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:22.206083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4c6db4b3-0121-4116-9bf3-6cb50e40992f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:22.239091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38910c0-3cb4-44ff-b3f6-dbc75b608a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:22.239770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38910c0-3cb4-44ff-b3f6-dbc75b608a52 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:22.240051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38910c0-3cb4-44ff-b3f6-dbc75b608a52 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:22.240322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38910c0-3cb4-44ff-b3f6-dbc75b608a52 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:22.241002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38910c0-3cb4-44ff-b3f6-dbc75b608a52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:22.242667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38910c0-3cb4-44ff-b3f6-dbc75b608a52 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:22.249415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 395/776] 10.222.0.121 () {56 vars in 1092 bytes} [Thu May 26 17:40:22 2022] GET /identity/v3/services => generated 776 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:22.250256 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38910c0-3cb4-44ff-b3f6-dbc75b608a52 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:22.307075 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:22.316788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 382/777] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:22 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:22.340105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6db4b3-0121-4116-9bf3-6cb50e40992f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:22.375155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:22.471458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6db4b3-0121-4116-9bf3-6cb50e40992f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:22.501948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38910c0-3cb4-44ff-b3f6-dbc75b608a52 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:22.547161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a757523-c4fa-413c-a720-c21797dfc399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:22.548402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a757523-c4fa-413c-a720-c21797dfc399 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:22.548791 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a757523-c4fa-413c-a720-c21797dfc399 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:22.548968 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a757523-c4fa-413c-a720-c21797dfc399 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:22.549960 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a757523-c4fa-413c-a720-c21797dfc399 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:22.551172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a757523-c4fa-413c-a720-c21797dfc399 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:22.563467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a94e1e-32fa-4219-aff0-7b1afdf707b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-1a757523-c4fa-413c-a720-c21797dfc399 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:22.564391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:22.565319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:22.567767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 396/778] 10.222.0.121 () {56 vars in 1134 bytes} [Thu May 26 17:40:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:22.569375 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a94e1e-32fa-4219-aff0-7b1afdf707b1 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:22.569859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a94e1e-32fa-4219-aff0-7b1afdf707b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:22.570329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a94e1e-32fa-4219-aff0-7b1afdf707b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:22.571646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a757523-c4fa-413c-a720-c21797dfc399 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:22.588650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a94e1e-32fa-4219-aff0-7b1afdf707b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-00a94e1e-32fa-4219-aff0-7b1afdf707b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:22.597572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:22.598551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:22.599438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:22.599438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:22.599438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:22.599438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:22.599438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:22.599438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:22.599438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:22.599438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:22.599438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:22.599438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:22.599438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:22.599438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:22.602222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 383/779] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:40:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:22.607602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a94e1e-32fa-4219-aff0-7b1afdf707b1 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:22.669364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:22.685717 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a757523-c4fa-413c-a720-c21797dfc399 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:22.706722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:22.724708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a94e1e-32fa-4219-aff0-7b1afdf707b1 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:22.724970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6d8f9-3378-4d66-ac0e-59a1c29e587f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:22.725635 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6d8f9-3378-4d66-ac0e-59a1c29e587f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:22.725921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6d8f9-3378-4d66-ac0e-59a1c29e587f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:22.726197 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6d8f9-3378-4d66-ac0e-59a1c29e587f None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:22.726946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6d8f9-3378-4d66-ac0e-59a1c29e587f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:22.727254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6d8f9-3378-4d66-ac0e-59a1c29e587f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:22.728629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6d8f9-3378-4d66-ac0e-59a1c29e587f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:22.733877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 397/780] 10.222.0.121 () {56 vars in 1129 bytes} [Thu May 26 17:40:22 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:22.740104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6d8f9-3378-4d66-ac0e-59a1c29e587f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:22.765011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41257a-78d5-4b25-9fb4-668cdc5f8f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:22.766071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41257a-78d5-4b25-9fb4-668cdc5f8f36 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:22.766580 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41257a-78d5-4b25-9fb4-668cdc5f8f36 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:22.767255 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41257a-78d5-4b25-9fb4-668cdc5f8f36 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:22.768311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41257a-78d5-4b25-9fb4-668cdc5f8f36 None admin] RBAC: Adding query filter params (name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:22.768880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41257a-78d5-4b25-9fb4-668cdc5f8f36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:22.770451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41257a-78d5-4b25-9fb4-668cdc5f8f36 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:22.780390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 384/781] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:40:22 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:22.788161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41257a-78d5-4b25-9fb4-668cdc5f8f36 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:22.856826 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6d8f9-3378-4d66-ac0e-59a1c29e587f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:22.895772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b95478-a86b-4d55-8b48-5fb57428b428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:22.896451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b95478-a86b-4d55-8b48-5fb57428b428 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:22.896734 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b95478-a86b-4d55-8b48-5fb57428b428 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:22.897008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b95478-a86b-4d55-8b48-5fb57428b428 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:22.897832 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b95478-a86b-4d55-8b48-5fb57428b428 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:22.898143 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b95478-a86b-4d55-8b48-5fb57428b428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:22.899727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b95478-a86b-4d55-8b48-5fb57428b428 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:22.905397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 398/782] 10.222.0.121 () {56 vars in 1129 bytes} [Thu May 26 17:40:22 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:22.911965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b95478-a86b-4d55-8b48-5fb57428b428 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:22.931234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:22.990353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41257a-78d5-4b25-9fb4-668cdc5f8f36 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:23.048256 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4b4d8-c207-4f8d-8dd1-1588c62a5dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:23.048472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b95478-a86b-4d55-8b48-5fb57428b428 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:23.052439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4b4d8-c207-4f8d-8dd1-1588c62a5dba None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:23.052739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4b4d8-c207-4f8d-8dd1-1588c62a5dba None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:23.053013 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4b4d8-c207-4f8d-8dd1-1588c62a5dba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:23.072143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4b4d8-c207-4f8d-8dd1-1588c62a5dba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:23.073606 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4b4d8-c207-4f8d-8dd1-1588c62a5dba None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:23.089956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-23d4b4d8-c207-4f8d-8dd1-1588c62a5dba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:23.111824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:23.113864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:23.115384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:23.115384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:23.115384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:23.115384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:23.120324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 385/783] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:23.124205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3d4c5-71b8-4ad6-ac61-f20645c515d1 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:23.124848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3d4c5-71b8-4ad6-ac61-f20645c515d1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:23.125269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3d4c5-71b8-4ad6-ac61-f20645c515d1 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:23.126857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976299e1-127c-46d8-99c5-bebe3f9c3baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:23.127803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976299e1-127c-46d8-99c5-bebe3f9c3baf None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:23.128297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976299e1-127c-46d8-99c5-bebe3f9c3baf None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:23.133096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976299e1-127c-46d8-99c5-bebe3f9c3baf None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:23.133096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976299e1-127c-46d8-99c5-bebe3f9c3baf None admin] RBAC: Adding query filter params (interface=public, service_id=d32702d1ea854ef4846f588a022a9958, region_id=RegionOne) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:23.133096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976299e1-127c-46d8-99c5-bebe3f9c3baf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:23.133096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976299e1-127c-46d8-99c5-bebe3f9c3baf None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:23.133633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 386/784] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:23 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:23.135240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3d4c5-71b8-4ad6-ac61-f20645c515d1 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:23.140478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 399/785] 10.222.0.121 () {56 vars in 1256 bytes} [Thu May 26 17:40:22 2022] GET /identity/v3/endpoints?service_id=d32702d1ea854ef4846f588a022a9958&interface=public®ion_id=RegionOne => generated 195 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:23.142500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f9c6c-2252-4d05-8e1c-8739e5e06d35 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:23.142964 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f9c6c-2252-4d05-8e1c-8739e5e06d35 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:23.143319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f9c6c-2252-4d05-8e1c-8739e5e06d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:23.172725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c94f9c6c-2252-4d05-8e1c-8739e5e06d35 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:23.177970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c94f9c6c-2252-4d05-8e1c-8739e5e06d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:23.286725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 400/786] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:23 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:23.286725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f9c6c-2252-4d05-8e1c-8739e5e06d35 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:23.294664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3d4c5-71b8-4ad6-ac61-f20645c515d1 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:23.300248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:23.363262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3b000-430c-4e7a-9b35-1fb43bdd07c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:23.363978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3b000-430c-4e7a-9b35-1fb43bdd07c8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:23.364218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3b000-430c-4e7a-9b35-1fb43bdd07c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:23.364488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3b000-430c-4e7a-9b35-1fb43bdd07c8 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:23.365217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3b000-430c-4e7a-9b35-1fb43bdd07c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:23.365524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3b000-430c-4e7a-9b35-1fb43bdd07c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:23.367295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3b000-430c-4e7a-9b35-1fb43bdd07c8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:23.381201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 387/787] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:23.386904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3b000-430c-4e7a-9b35-1fb43bdd07c8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:23.432418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f9c6c-2252-4d05-8e1c-8739e5e06d35 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:23.480376 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:23.482055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa676f-c16b-456d-9cea-61e02253ac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:23.482767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa676f-c16b-456d-9cea-61e02253ac3b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:23.484013 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa676f-c16b-456d-9cea-61e02253ac3b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:23.484294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa676f-c16b-456d-9cea-61e02253ac3b None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:23.486935 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa676f-c16b-456d-9cea-61e02253ac3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:23.489927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa676f-c16b-456d-9cea-61e02253ac3b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:23.505464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 401/788] 10.222.0.121 () {56 vars in 1092 bytes} [Thu May 26 17:40:23 2022] GET /identity/v3/services => generated 776 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:23.505464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa676f-c16b-456d-9cea-61e02253ac3b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:23.618624 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3b000-430c-4e7a-9b35-1fb43bdd07c8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:23.685684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:23.699472 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190a0b7-0adf-4258-8be0-ca1753ed8091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:23.700288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190a0b7-0adf-4258-8be0-ca1753ed8091 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:23.700571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190a0b7-0adf-4258-8be0-ca1753ed8091 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:23.700844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190a0b7-0adf-4258-8be0-ca1753ed8091 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:23.723071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190a0b7-0adf-4258-8be0-ca1753ed8091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-1190a0b7-0adf-4258-8be0-ca1753ed8091 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:23.733765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:23.741097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:23.741930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 388/789] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:40:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:23.749977 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa676f-c16b-456d-9cea-61e02253ac3b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:23.750182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190a0b7-0adf-4258-8be0-ca1753ed8091 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:23.797225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054dead-02c4-4c0f-a40e-36ef23cc3319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:23.799166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054dead-02c4-4c0f-a40e-36ef23cc3319 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:23.799871 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054dead-02c4-4c0f-a40e-36ef23cc3319 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:23.800859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054dead-02c4-4c0f-a40e-36ef23cc3319 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:23.810219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054dead-02c4-4c0f-a40e-36ef23cc3319 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:23.812892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054dead-02c4-4c0f-a40e-36ef23cc3319 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-0054dead-02c4-4c0f-a40e-36ef23cc3319 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:23.835060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:23.836049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:23.836990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:23.836990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:23.836990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:23.836990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:23.836990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:23.836990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:23.836990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:23.836990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:23.836990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:40:23.836990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:23.836990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 402/790] 10.222.0.121 () {56 vars in 1122 bytes} [Thu May 26 17:40:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:23.842011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054dead-02c4-4c0f-a40e-36ef23cc3319 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:23.894019 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190a0b7-0adf-4258-8be0-ca1753ed8091 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:23.923763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:23.943376 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a5157-f8d9-40ff-bc41-420d7757deb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:23.944983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a5157-f8d9-40ff-bc41-420d7757deb2 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:23.945475 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a5157-f8d9-40ff-bc41-420d7757deb2 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:23.945953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a5157-f8d9-40ff-bc41-420d7757deb2 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:23.946930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a5157-f8d9-40ff-bc41-420d7757deb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:23.947448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a5157-f8d9-40ff-bc41-420d7757deb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:23.949879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a5157-f8d9-40ff-bc41-420d7757deb2 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:23.968900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 389/791] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:40:23 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:23.975109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a5157-f8d9-40ff-bc41-420d7757deb2 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:23.978069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054dead-02c4-4c0f-a40e-36ef23cc3319 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:24.011996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:24.025192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da484f2-3c4d-4e9d-83e4-c4ec406bc465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:24.027641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da484f2-3c4d-4e9d-83e4-c4ec406bc465 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:24.027641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da484f2-3c4d-4e9d-83e4-c4ec406bc465 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:24.027641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da484f2-3c4d-4e9d-83e4-c4ec406bc465 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:24.027641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da484f2-3c4d-4e9d-83e4-c4ec406bc465 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:24.027641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da484f2-3c4d-4e9d-83e4-c4ec406bc465 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:24.028674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da484f2-3c4d-4e9d-83e4-c4ec406bc465 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:24.066898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 403/792] 10.222.0.121 () {56 vars in 1154 bytes} [Thu May 26 17:40:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:24.068118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da484f2-3c4d-4e9d-83e4-c4ec406bc465 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:24.099014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a5157-f8d9-40ff-bc41-420d7757deb2 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:24.136375 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce71be-2c37-41cf-befa-7fb290f03da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:24.137592 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce71be-2c37-41cf-befa-7fb290f03da1 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:24.138077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce71be-2c37-41cf-befa-7fb290f03da1 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:24.138572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce71be-2c37-41cf-befa-7fb290f03da1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:24.150399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce71be-2c37-41cf-befa-7fb290f03da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:24.152188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce71be-2c37-41cf-befa-7fb290f03da1 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-3bce71be-2c37-41cf-befa-7fb290f03da1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:24.172078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:24.173114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:24.174002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:24.174002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:24.174002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:24.174002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:24.176387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 390/793] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:24.182893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da484f2-3c4d-4e9d-83e4-c4ec406bc465 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:24.184039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce71be-2c37-41cf-befa-7fb290f03da1 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:24.226499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99b508-ac05-400d-8198-1e73a4a11a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:24.227344 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99b508-ac05-400d-8198-1e73a4a11a58 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:24.227621 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99b508-ac05-400d-8198-1e73a4a11a58 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:24.227889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99b508-ac05-400d-8198-1e73a4a11a58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:24.250202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:24.253418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99b508-ac05-400d-8198-1e73a4a11a58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:24.259707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99b508-ac05-400d-8198-1e73a4a11a58 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7a99b508-ac05-400d-8198-1e73a4a11a58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:24.288880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:24.290264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:24.291446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:24.291446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:24.291446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:24.291446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:24.291446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 404/794] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:24.303433 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99b508-ac05-400d-8198-1e73a4a11a58 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:24.372997 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce71be-2c37-41cf-befa-7fb290f03da1 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:24.393392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:24.424297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9916d52-8f7b-4744-9629-d184c13e78a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:24.424929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9916d52-8f7b-4744-9629-d184c13e78a6 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:24.425207 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9916d52-8f7b-4744-9629-d184c13e78a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:24.425478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9916d52-8f7b-4744-9629-d184c13e78a6 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:24.426202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9916d52-8f7b-4744-9629-d184c13e78a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:24.426506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9916d52-8f7b-4744-9629-d184c13e78a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:24.427905 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9916d52-8f7b-4744-9629-d184c13e78a6 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:24.446303 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 391/795] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:24.451846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9916d52-8f7b-4744-9629-d184c13e78a6 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:24.486295 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99b508-ac05-400d-8198-1e73a4a11a58 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:24.546455 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf921f2-0377-490d-bc65-0939ac7b3bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:24.548142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf921f2-0377-490d-bc65-0939ac7b3bcb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:24.548636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf921f2-0377-490d-bc65-0939ac7b3bcb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:24.549112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf921f2-0377-490d-bc65-0939ac7b3bcb None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:24.550068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf921f2-0377-490d-bc65-0939ac7b3bcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:24.550574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf921f2-0377-490d-bc65-0939ac7b3bcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:24.552203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf921f2-0377-490d-bc65-0939ac7b3bcb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:24.583286 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 405/796] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:24.584832 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf921f2-0377-490d-bc65-0939ac7b3bcb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:24.593474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9916d52-8f7b-4744-9629-d184c13e78a6 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:24.598093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:24.670767 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828288b-535d-4dde-92f5-a23cd3eeead8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:24.671591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828288b-535d-4dde-92f5-a23cd3eeead8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:24.671865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828288b-535d-4dde-92f5-a23cd3eeead8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:24.672129 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828288b-535d-4dde-92f5-a23cd3eeead8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:24.685999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828288b-535d-4dde-92f5-a23cd3eeead8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:24.687405 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828288b-535d-4dde-92f5-a23cd3eeead8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-0828288b-535d-4dde-92f5-a23cd3eeead8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:24.716184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:24.717225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:24.718247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:24.718247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:24.718247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:24.718247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:24.718247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:24.718247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:24.718247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:24.718247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:24.718247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 17:40:24.718247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:24.724851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 392/797] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:40:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:24.728921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828288b-535d-4dde-92f5-a23cd3eeead8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:24.738173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf921f2-0377-490d-bc65-0939ac7b3bcb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:24.740054 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:24.792956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce7536-35f0-4014-af73-e70f49fd41a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:24.793767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce7536-35f0-4014-af73-e70f49fd41a7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:24.794040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce7536-35f0-4014-af73-e70f49fd41a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:24.794326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce7536-35f0-4014-af73-e70f49fd41a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:24.809343 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce7536-35f0-4014-af73-e70f49fd41a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:24.812672 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce7536-35f0-4014-af73-e70f49fd41a7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-01ce7536-35f0-4014-af73-e70f49fd41a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:24.830703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:24.831666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:24.832539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:24.832539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:24.832539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:24.832539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:24.832539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:24.832539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:24.832539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:24.832539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:24.832539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 406/798] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:24.833492 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885b0eb-99de-45dd-90a7-df47cfe4260c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:24.833779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885b0eb-99de-45dd-90a7-df47cfe4260c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:24.834057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885b0eb-99de-45dd-90a7-df47cfe4260c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:24.839386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 407/799] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:24.840192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885b0eb-99de-45dd-90a7-df47cfe4260c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:24.881538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828288b-535d-4dde-92f5-a23cd3eeead8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:24.917996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e4df7-40f0-42c6-8445-2f3e4bc02f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:24.919015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e4df7-40f0-42c6-8445-2f3e4bc02f68 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:24.919506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e4df7-40f0-42c6-8445-2f3e4bc02f68 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:24.919965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e4df7-40f0-42c6-8445-2f3e4bc02f68 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:24.920978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e4df7-40f0-42c6-8445-2f3e4bc02f68 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:24.921557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e4df7-40f0-42c6-8445-2f3e4bc02f68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:24.923839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e4df7-40f0-42c6-8445-2f3e4bc02f68 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:24.946970 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 393/800] 10.222.0.121 () {56 vars in 1140 bytes} [Thu May 26 17:40:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:24.948634 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee14197-6941-4a29-bf11-ecbde58260d4 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:24.949105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee14197-6941-4a29-bf11-ecbde58260d4 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:24.949583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee14197-6941-4a29-bf11-ecbde58260d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:24.967529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-eee14197-6941-4a29-bf11-ecbde58260d4 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:24.968346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885b0eb-99de-45dd-90a7-df47cfe4260c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:24.969432 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-eee14197-6941-4a29-bf11-ecbde58260d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:24.978122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:25.010336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb15436-516c-4c2d-9e01-96ba8ac651ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:25.011042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb15436-516c-4c2d-9e01-96ba8ac651ee None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:25.011329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb15436-516c-4c2d-9e01-96ba8ac651ee None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:25.011603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb15436-516c-4c2d-9e01-96ba8ac651ee None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:25.012404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb15436-516c-4c2d-9e01-96ba8ac651ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:25.012697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb15436-516c-4c2d-9e01-96ba8ac651ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:25.014568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb15436-516c-4c2d-9e01-96ba8ac651ee None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:25.032044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 408/801] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:25.033174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb15436-516c-4c2d-9e01-96ba8ac651ee None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:25.053765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 394/802] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:24 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:25.055178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee14197-6941-4a29-bf11-ecbde58260d4 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:25.082517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:25.180105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb15436-516c-4c2d-9e01-96ba8ac651ee None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:25.192772 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee14197-6941-4a29-bf11-ecbde58260d4 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:25.219665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213aa32e-4aa3-4f3b-9ded-553a5a1ca6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:25.220727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aa32e-4aa3-4f3b-9ded-553a5a1ca6d2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:25.221252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aa32e-4aa3-4f3b-9ded-553a5a1ca6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:25.221666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aa32e-4aa3-4f3b-9ded-553a5a1ca6d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:25.229934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a829d7b-7efc-43a0-97f1-c5708843b8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:25.231178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a829d7b-7efc-43a0-97f1-c5708843b8b0 None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:25.231684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a829d7b-7efc-43a0-97f1-c5708843b8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:25.232185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a829d7b-7efc-43a0-97f1-c5708843b8b0 None admin] PATH_INFO: `/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/485c784a043541619ffffc657b21ab97/roles/9399fac5b71b46f79d1ebe5142fc1176` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:25.233076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213aa32e-4aa3-4f3b-9ded-553a5a1ca6d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:25.235440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213aa32e-4aa3-4f3b-9ded-553a5a1ca6d2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:25.245963 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:25.249881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a829d7b-7efc-43a0-97f1-c5708843b8b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61dea65b34b47a186151c6a63bc7d79, user_id=485c784a043541619ffffc657b21ab97, role_id=9399fac5b71b46f79d1ebe5142fc1176)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:25.251817 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a829d7b-7efc-43a0-97f1-c5708843b8b0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-213aa32e-4aa3-4f3b-9ded-553a5a1ca6d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:25.256207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:25.262764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:25.263747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:25.263747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:25.263747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:25.263747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:25.263747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 409/803] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:25.263747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213aa32e-4aa3-4f3b-9ded-553a5a1ca6d2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:25.343174 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 395/804] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:40:25 2022] PUT /identity/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/485c784a043541619ffffc657b21ab97/roles/9399fac5b71b46f79d1ebe5142fc1176 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:40:25.343595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:25.346560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea73898-9f4d-4aa7-9c4c-0fc5635f131f None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:25.347056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea73898-9f4d-4aa7-9c4c-0fc5635f131f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:25.347493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea73898-9f4d-4aa7-9c4c-0fc5635f131f None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:25.354793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 396/805] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:25 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:25.355927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea73898-9f4d-4aa7-9c4c-0fc5635f131f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:25.383973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213aa32e-4aa3-4f3b-9ded-553a5a1ca6d2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:25.431868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4091bf-2ad8-40e7-819c-5a987eaa886a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:25.433173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4091bf-2ad8-40e7-819c-5a987eaa886a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:25.433651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4091bf-2ad8-40e7-819c-5a987eaa886a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:25.434131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4091bf-2ad8-40e7-819c-5a987eaa886a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:25.448398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4091bf-2ad8-40e7-819c-5a987eaa886a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-8b4091bf-2ad8-40e7-819c-5a987eaa886a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:25.458030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:25.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:25.459917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:25.459917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:25.459917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:25.459917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:25.459917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:25.459917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:25.459917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:25.459917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:25.459917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:25.459917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:25.459917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:40:25.459917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:25.462866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 410/806] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:40:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:25.464510 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952ff59-55ed-4e06-b9b2-a69f386d6da9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:25.465025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952ff59-55ed-4e06-b9b2-a69f386d6da9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:25.465500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952ff59-55ed-4e06-b9b2-a69f386d6da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:25.485839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a952ff59-55ed-4e06-b9b2-a69f386d6da9 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:25.487692 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a952ff59-55ed-4e06-b9b2-a69f386d6da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:25.511584 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea73898-9f4d-4aa7-9c4c-0fc5635f131f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:25.558062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e2f56-845a-49e4-a52c-7281d5a55eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:25.559048 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e2f56-845a-49e4-a52c-7281d5a55eff None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:25.559517 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e2f56-845a-49e4-a52c-7281d5a55eff None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:25.560004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e2f56-845a-49e4-a52c-7281d5a55eff None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:25.560875 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e2f56-845a-49e4-a52c-7281d5a55eff None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:25.561386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e2f56-845a-49e4-a52c-7281d5a55eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:25.562758 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e2f56-845a-49e4-a52c-7281d5a55eff None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:25.576046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 397/807] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:25.576970 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e2f56-845a-49e4-a52c-7281d5a55eff None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:25.578527 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 411/808] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:25 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:25.585634 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952ff59-55ed-4e06-b9b2-a69f386d6da9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:25.660108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:25.695297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:25.700575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952ff59-55ed-4e06-b9b2-a69f386d6da9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:25.713977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e2f56-845a-49e4-a52c-7281d5a55eff None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:25.739404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f54262-663b-4f74-bfc4-4960e7b532c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:25.740129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f54262-663b-4f74-bfc4-4960e7b532c6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:25.740464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f54262-663b-4f74-bfc4-4960e7b532c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:25.740737 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f54262-663b-4f74-bfc4-4960e7b532c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:25.749088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f54262-663b-4f74-bfc4-4960e7b532c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:25.750662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f54262-663b-4f74-bfc4-4960e7b532c6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:25.756179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27324c62-3354-41ee-8b7c-1f7ee951c3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:25.756871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27324c62-3354-41ee-8b7c-1f7ee951c3ab None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:25.757157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27324c62-3354-41ee-8b7c-1f7ee951c3ab None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:25.757603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27324c62-3354-41ee-8b7c-1f7ee951c3ab None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:25.758487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27324c62-3354-41ee-8b7c-1f7ee951c3ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:25.758804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27324c62-3354-41ee-8b7c-1f7ee951c3ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:25.760248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27324c62-3354-41ee-8b7c-1f7ee951c3ab None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f7f54262-663b-4f74-bfc4-4960e7b532c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:25.768606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:25.769673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:25.770773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:25.770773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:25.770773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:25.770773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:25.770773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:25.770773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:25.770773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:25.770773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:25.772673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 412/810] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:25.773914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f54262-663b-4f74-bfc4-4960e7b532c6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:25.774909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 398/810] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:40:25 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:25.776018 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27324c62-3354-41ee-8b7c-1f7ee951c3ab None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:25.906286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27324c62-3354-41ee-8b7c-1f7ee951c3ab None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:25.910094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f54262-663b-4f74-bfc4-4960e7b532c6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:25.923262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:25.943694 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:25.945611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf50f6-a368-4350-8374-c1ffb41bcd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:25.946400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf50f6-a368-4350-8374-c1ffb41bcd09 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:25.946688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf50f6-a368-4350-8374-c1ffb41bcd09 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:25.947011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf50f6-a368-4350-8374-c1ffb41bcd09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:25.957361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eead9e3-e541-4a23-95fc-5256c35e10f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:25.958667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eead9e3-e541-4a23-95fc-5256c35e10f8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:25.959003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf50f6-a368-4350-8374-c1ffb41bcd09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:25.959385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eead9e3-e541-4a23-95fc-5256c35e10f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:25.959798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eead9e3-e541-4a23-95fc-5256c35e10f8 None admin] PATH_INFO: `/v3/services/network` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:25.960588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eead9e3-e541-4a23-95fc-5256c35e10f8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:25.960877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf50f6-a368-4350-8374-c1ffb41bcd09 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:25.962246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eead9e3-e541-4a23-95fc-5256c35e10f8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-9eead9e3-e541-4a23-95fc-5256c35e10f8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:25.976868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:25.978057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:40:25.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-0dcf50f6-a368-4350-8374-c1ffb41bcd09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:25.980628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:25.981627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:25.983126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:25.983126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:25.983126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:25.983126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:25.984516 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 413/811] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:25 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:25.986444 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eead9e3-e541-4a23-95fc-5256c35e10f8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:25.990140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 399/812] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:25.991117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf50f6-a368-4350-8374-c1ffb41bcd09 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:26.109416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf50f6-a368-4350-8374-c1ffb41bcd09 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:26.133253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eead9e3-e541-4a23-95fc-5256c35e10f8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:26.145898 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1d6c7-baf5-4769-8e66-f4333880d4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:26.146672 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1d6c7-baf5-4769-8e66-f4333880d4fe None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:26.147057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1d6c7-baf5-4769-8e66-f4333880d4fe None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:26.147407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1d6c7-baf5-4769-8e66-f4333880d4fe None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:26.148216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1d6c7-baf5-4769-8e66-f4333880d4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:26.148599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1d6c7-baf5-4769-8e66-f4333880d4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:26.150097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1d6c7-baf5-4769-8e66-f4333880d4fe None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:26.167192 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 400/813] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:26.168205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1d6c7-baf5-4769-8e66-f4333880d4fe None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:26.178387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcb89e-f98f-4594-a7ad-adab60c786eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:26.179369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcb89e-f98f-4594-a7ad-adab60c786eb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:26.180383 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcb89e-f98f-4594-a7ad-adab60c786eb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:26.180838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcb89e-f98f-4594-a7ad-adab60c786eb None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:26.182249 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcb89e-f98f-4594-a7ad-adab60c786eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:26.182735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcb89e-f98f-4594-a7ad-adab60c786eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:26.185108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcb89e-f98f-4594-a7ad-adab60c786eb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:26.201959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:26.203042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 414/814] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:26.204144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcb89e-f98f-4594-a7ad-adab60c786eb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:26.257967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:26.289718 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1d6c7-baf5-4769-8e66-f4333880d4fe None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:26.317307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcb89e-f98f-4594-a7ad-adab60c786eb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:26.327928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6cf1e-b1d4-43f9-9e24-6bd6ddeea082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:26.329240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6cf1e-b1d4-43f9-9e24-6bd6ddeea082 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:26.329579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6cf1e-b1d4-43f9-9e24-6bd6ddeea082 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:26.329887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6cf1e-b1d4-43f9-9e24-6bd6ddeea082 None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:26.330503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6cf1e-b1d4-43f9-9e24-6bd6ddeea082 None admin] RBAC: Adding query filter params (name=network) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:26.330812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6cf1e-b1d4-43f9-9e24-6bd6ddeea082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:26.332211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6cf1e-b1d4-43f9-9e24-6bd6ddeea082 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:26.341282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 401/815] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:26 2022] GET /identity/v3/services?name=network => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:26.341282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfc61b-79cb-40a0-8baf-ad9a9b2b6f64 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:26.341282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfc61b-79cb-40a0-8baf-ad9a9b2b6f64 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:26.341282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfc61b-79cb-40a0-8baf-ad9a9b2b6f64 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:26.341282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 402/816] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:26 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:26.341282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfc61b-79cb-40a0-8baf-ad9a9b2b6f64 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:26.357767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cdc13-9aae-43fc-b66a-39e85b2f022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:26.358562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cdc13-9aae-43fc-b66a-39e85b2f022c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:26.358863 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cdc13-9aae-43fc-b66a-39e85b2f022c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:26.359139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cdc13-9aae-43fc-b66a-39e85b2f022c None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:26.367110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cdc13-9aae-43fc-b66a-39e85b2f022c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:26.368541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cdc13-9aae-43fc-b66a-39e85b2f022c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f83cdc13-9aae-43fc-b66a-39e85b2f022c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:26.383303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:26.384291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:26.385949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:26.385949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:26.385949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:26.385949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:26.385949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:26.385949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:26.385949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:26.385949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:26.385949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:40:26.385949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:26.385949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 415/817] 10.222.0.121 () {56 vars in 1098 bytes} [Thu May 26 17:40:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:26.385949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cdc13-9aae-43fc-b66a-39e85b2f022c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:26.444898 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:26.456844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfc61b-79cb-40a0-8baf-ad9a9b2b6f64 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:26.494171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cdc13-9aae-43fc-b66a-39e85b2f022c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:26.500873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e9600-6da7-47bc-b723-7023c5b646c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:26.501858 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e9600-6da7-47bc-b723-7023c5b646c5 None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:26.502687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e9600-6da7-47bc-b723-7023c5b646c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:26.502997 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e9600-6da7-47bc-b723-7023c5b646c5 None admin] PATH_INFO: `/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/eb9ba9c73956403ba870f6f6b0e687aa/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:26.504731 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:26.506145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e9600-6da7-47bc-b723-7023c5b646c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61dea65b34b47a186151c6a63bc7d79, user_id=eb9ba9c73956403ba870f6f6b0e687aa, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:26.507644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e9600-6da7-47bc-b723-7023c5b646c5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:26.530337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419812c-b6e9-4335-8fdd-84108f176fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:26.531010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419812c-b6e9-4335-8fdd-84108f176fd0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:26.531886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419812c-b6e9-4335-8fdd-84108f176fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:26.536884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419812c-b6e9-4335-8fdd-84108f176fd0 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:26.536884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419812c-b6e9-4335-8fdd-84108f176fd0 None admin] RBAC: Adding query filter params (type=network) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:26.536884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419812c-b6e9-4335-8fdd-84108f176fd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:26.536884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419812c-b6e9-4335-8fdd-84108f176fd0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:26.541304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 416/818] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:26 2022] GET /identity/v3/services?type=network => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:26.542514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ce9d0-c73c-48fe-87b4-3666311b9db3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:26.542796 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ce9d0-c73c-48fe-87b4-3666311b9db3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:26.543100 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ce9d0-c73c-48fe-87b4-3666311b9db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:26.560871 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-773ce9d0-c73c-48fe-87b4-3666311b9db3 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:26.562186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-773ce9d0-c73c-48fe-87b4-3666311b9db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:26.590066 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 403/819] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:40:26 2022] PUT /identity/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/eb9ba9c73956403ba870f6f6b0e687aa/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:40:26.592181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e9600-6da7-47bc-b723-7023c5b646c5 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:26.643476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 417/820] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:26 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:26.644408 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ce9d0-c73c-48fe-87b4-3666311b9db3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:26.735813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e9600-6da7-47bc-b723-7023c5b646c5 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:26.776737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab128c-01b4-4bed-99f9-8680c9d25717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:26.809697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab128c-01b4-4bed-99f9-8680c9d25717 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:26.809697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab128c-01b4-4bed-99f9-8680c9d25717 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:26.809697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab128c-01b4-4bed-99f9-8680c9d25717 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:26.809697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab128c-01b4-4bed-99f9-8680c9d25717 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:26.809697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab128c-01b4-4bed-99f9-8680c9d25717 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:26.809697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab128c-01b4-4bed-99f9-8680c9d25717 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:26.810233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ce9d0-c73c-48fe-87b4-3666311b9db3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:26.810233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:26.810390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 404/821] 10.222.0.121 () {56 vars in 1138 bytes} [Thu May 26 17:40:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:26.810390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab128c-01b4-4bed-99f9-8680c9d25717 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:26.821676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:26.833209 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f4af9-4d43-4212-95a6-7b11fbbea814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:26.833940 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f4af9-4d43-4212-95a6-7b11fbbea814 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:26.834228 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f4af9-4d43-4212-95a6-7b11fbbea814 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:26.834500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f4af9-4d43-4212-95a6-7b11fbbea814 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:26.835618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f4af9-4d43-4212-95a6-7b11fbbea814 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:26.836667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f4af9-4d43-4212-95a6-7b11fbbea814 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:26.926384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab128c-01b4-4bed-99f9-8680c9d25717 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:26.960671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59a082-dcb1-41a3-af4f-d88ad854a437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:26.961481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59a082-dcb1-41a3-af4f-d88ad854a437 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:26.961758 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59a082-dcb1-41a3-af4f-d88ad854a437 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:26.962030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59a082-dcb1-41a3-af4f-d88ad854a437 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:26.962670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59a082-dcb1-41a3-af4f-d88ad854a437 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:26.964122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59a082-dcb1-41a3-af4f-d88ad854a437 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-af59a082-dcb1-41a3-af4f-d88ad854a437 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:26.979357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:26.980410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:26.981316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 405/822] 10.222.0.121 () {56 vars in 1134 bytes} [Thu May 26 17:40:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:26.982229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59a082-dcb1-41a3-af4f-d88ad854a437 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:27.085040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59a082-dcb1-41a3-af4f-d88ad854a437 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:27.122850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 418/823] 10.222.0.121 () {60 vars in 1149 bytes} [Thu May 26 17:40:26 2022] POST /identity/v3/endpoints => generated 329 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:40:27.123906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f4af9-4d43-4212-95a6-7b11fbbea814 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:27.151938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248422e-d1b8-43bf-9886-d56630a655e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:27.153637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248422e-d1b8-43bf-9886-d56630a655e4 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:27.154164 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248422e-d1b8-43bf-9886-d56630a655e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:27.154694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248422e-d1b8-43bf-9886-d56630a655e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:27.205945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:27.292638 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f4af9-4d43-4212-95a6-7b11fbbea814 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:27.316069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248422e-d1b8-43bf-9886-d56630a655e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:27.317659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248422e-d1b8-43bf-9886-d56630a655e4 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:27.338239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088534df-2514-4234-8258-76fb4dd67df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:27.339297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088534df-2514-4234-8258-76fb4dd67df8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:27.339809 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088534df-2514-4234-8258-76fb4dd67df8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:27.340323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088534df-2514-4234-8258-76fb4dd67df8 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:27.341235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088534df-2514-4234-8258-76fb4dd67df8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:27.341754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088534df-2514-4234-8258-76fb4dd67df8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:27.343371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088534df-2514-4234-8258-76fb4dd67df8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:27.349303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 419/824] 10.222.0.121 () {56 vars in 1129 bytes} [Thu May 26 17:40:27 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:27.354982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088534df-2514-4234-8258-76fb4dd67df8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:27.382855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:27.431268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:27.459400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 406/825] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:26 2022] GET /identity/v3/auth/tokens => generated 1153 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:40:27.465429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248422e-d1b8-43bf-9886-d56630a655e4 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:27.474582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088534df-2514-4234-8258-76fb4dd67df8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:27.513344 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836ce0e-3b0d-4c96-aa5a-c346ce2ff386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:27.514988 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836ce0e-3b0d-4c96-aa5a-c346ce2ff386 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:27.515812 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836ce0e-3b0d-4c96-aa5a-c346ce2ff386 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:27.517102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836ce0e-3b0d-4c96-aa5a-c346ce2ff386 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:27.518706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836ce0e-3b0d-4c96-aa5a-c346ce2ff386 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:27.519684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836ce0e-3b0d-4c96-aa5a-c346ce2ff386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:27.522679 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836ce0e-3b0d-4c96-aa5a-c346ce2ff386 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:27.529677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 420/826] 10.222.0.121 () {56 vars in 1129 bytes} [Thu May 26 17:40:27 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:27.536517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836ce0e-3b0d-4c96-aa5a-c346ce2ff386 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:27.589246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248422e-d1b8-43bf-9886-d56630a655e4 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:27.635086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7916599-95ce-4d31-b9fa-a793c1eeef72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:27.635913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7916599-95ce-4d31-b9fa-a793c1eeef72 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:27.636197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7916599-95ce-4d31-b9fa-a793c1eeef72 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:27.636470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7916599-95ce-4d31-b9fa-a793c1eeef72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:27.644065 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:27.647603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7916599-95ce-4d31-b9fa-a793c1eeef72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:27.649044 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7916599-95ce-4d31-b9fa-a793c1eeef72 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:27.662051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836ce0e-3b0d-4c96-aa5a-c346ce2ff386 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-f7916599-95ce-4d31-b9fa-a793c1eeef72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:27.668798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:27.670575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:27.672144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:27.672144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:27.672144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:27.672144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:27.674938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 407/827] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:27.676939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7916599-95ce-4d31-b9fa-a793c1eeef72 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:27.699631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:27.709434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2dfd7-17c1-4680-a93d-87a306cb1cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:27.710418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2dfd7-17c1-4680-a93d-87a306cb1cc8 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:27.710946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2dfd7-17c1-4680-a93d-87a306cb1cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:27.711431 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2dfd7-17c1-4680-a93d-87a306cb1cc8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:27.712472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2dfd7-17c1-4680-a93d-87a306cb1cc8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:27.714148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2dfd7-17c1-4680-a93d-87a306cb1cc8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:27.783159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7916599-95ce-4d31-b9fa-a793c1eeef72 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:27.787421 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 421/828] 10.222.0.121 () {60 vars in 1149 bytes} [Thu May 26 17:40:27 2022] POST /identity/v3/endpoints => generated 348 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:40:27.788753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd79b45-9038-4e9c-b65e-c90733a3a240 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:27.789040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd79b45-9038-4e9c-b65e-c90733a3a240 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:27.789315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd79b45-9038-4e9c-b65e-c90733a3a240 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:27.794879 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 422/829] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:27.796889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b6d45-5e83-48e6-8fca-9daad6e1e424 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:27.797198 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b6d45-5e83-48e6-8fca-9daad6e1e424 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:27.798093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b6d45-5e83-48e6-8fca-9daad6e1e424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:27.837561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a9e7f-a56d-4cd8-923e-9cedb9301c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:27.838496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a9e7f-a56d-4cd8-923e-9cedb9301c5f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:27.838979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a9e7f-a56d-4cd8-923e-9cedb9301c5f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:27.839469 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a9e7f-a56d-4cd8-923e-9cedb9301c5f None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:27.840343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a9e7f-a56d-4cd8-923e-9cedb9301c5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:27.840789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a9e7f-a56d-4cd8-923e-9cedb9301c5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:27.842391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a9e7f-a56d-4cd8-923e-9cedb9301c5f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:27.848015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-672b6d45-5e83-48e6-8fca-9daad6e1e424 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:27.849280 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-672b6d45-5e83-48e6-8fca-9daad6e1e424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:27.866402 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 408/830] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:27.867977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a9e7f-a56d-4cd8-923e-9cedb9301c5f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:27.951276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 423/831] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:27 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:27.975084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b6d45-5e83-48e6-8fca-9daad6e1e424 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:27.975310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:28.012053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a9e7f-a56d-4cd8-923e-9cedb9301c5f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:28.060300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:28.060507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e45e45-d3ae-4bdd-b9cc-0fde599b175f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:28.060921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e45e45-d3ae-4bdd-b9cc-0fde599b175f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:28.061209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e45e45-d3ae-4bdd-b9cc-0fde599b175f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:28.061482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e45e45-d3ae-4bdd-b9cc-0fde599b175f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:28.069631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e45e45-d3ae-4bdd-b9cc-0fde599b175f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:28.071727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e45e45-d3ae-4bdd-b9cc-0fde599b175f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-71e45e45-d3ae-4bdd-b9cc-0fde599b175f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:28.088182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:28.089326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:28.090464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:28.090464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:28.090464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:28.090464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:28.090464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:28.090464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:28.090464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:28.090464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:28.092695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 409/832] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:28.093197 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b6d45-5e83-48e6-8fca-9daad6e1e424 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:28.104409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e45e45-d3ae-4bdd-b9cc-0fde599b175f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:28.139522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafa887-1647-40ed-8fab-00b5273b0c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:28.140383 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafa887-1647-40ed-8fab-00b5273b0c03 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:28.140912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafa887-1647-40ed-8fab-00b5273b0c03 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:28.141291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafa887-1647-40ed-8fab-00b5273b0c03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:28.153195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafa887-1647-40ed-8fab-00b5273b0c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-dbafa887-1647-40ed-8fab-00b5273b0c03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:28.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:28.165485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:28.166767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:28.166767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:28.166767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:28.166767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:28.166767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:28.166767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:28.166767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:28.166767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:28.166767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:28.166767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:28.166767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:40:28.166767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:28.168326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 424/833] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:40:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:28.175831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafa887-1647-40ed-8fab-00b5273b0c03 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:28.249590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e45e45-d3ae-4bdd-b9cc-0fde599b175f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:28.284549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafa887-1647-40ed-8fab-00b5273b0c03 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:28.290419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8b7cd-cf17-4a9a-aad4-7191a8f7b79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:28.291641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8b7cd-cf17-4a9a-aad4-7191a8f7b79d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:28.292276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8b7cd-cf17-4a9a-aad4-7191a8f7b79d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:28.292766 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8b7cd-cf17-4a9a-aad4-7191a8f7b79d None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:28.293759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8b7cd-cf17-4a9a-aad4-7191a8f7b79d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:28.294348 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8b7cd-cf17-4a9a-aad4-7191a8f7b79d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:28.296186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8b7cd-cf17-4a9a-aad4-7191a8f7b79d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:28.307319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:28.312537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 410/834] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:28.318578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8b7cd-cf17-4a9a-aad4-7191a8f7b79d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:28.323392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:28.330174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fd9ae-8813-47a0-8efb-512c0a467ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:28.331626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fd9ae-8813-47a0-8efb-512c0a467ba6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:28.334704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fd9ae-8813-47a0-8efb-512c0a467ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:28.334704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fd9ae-8813-47a0-8efb-512c0a467ba6 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:28.334704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fd9ae-8813-47a0-8efb-512c0a467ba6 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:28.334704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fd9ae-8813-47a0-8efb-512c0a467ba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:28.335647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fd9ae-8813-47a0-8efb-512c0a467ba6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:28.347544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 425/835] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:40:28 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:28.352638 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fd9ae-8813-47a0-8efb-512c0a467ba6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:28.429393 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8b7cd-cf17-4a9a-aad4-7191a8f7b79d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:28.468618 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab044ee7-8dce-4891-8a85-141cf974dfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:28.470019 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab044ee7-8dce-4891-8a85-141cf974dfb5 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:28.470370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab044ee7-8dce-4891-8a85-141cf974dfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:28.470830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab044ee7-8dce-4891-8a85-141cf974dfb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:28.471754 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab044ee7-8dce-4891-8a85-141cf974dfb5 None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=c0889a69a018452d8de1924072739d17) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:28.472138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab044ee7-8dce-4891-8a85-141cf974dfb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:28.473654 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab044ee7-8dce-4891-8a85-141cf974dfb5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:28.476617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fd9ae-8813-47a0-8efb-512c0a467ba6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:28.487257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 411/836] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:28 2022] GET /identity/v3/role_assignments?user.id=c0889a69a018452d8de1924072739d17&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=4f574961798246a4b12ad2276e67685d => generated 610 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:28.521775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d33f84-5189-473d-95a9-81beeecc32da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:28.522547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d33f84-5189-473d-95a9-81beeecc32da None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:28.522859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d33f84-5189-473d-95a9-81beeecc32da None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:28.523152 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d33f84-5189-473d-95a9-81beeecc32da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:28.536055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d33f84-5189-473d-95a9-81beeecc32da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:28.537548 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d33f84-5189-473d-95a9-81beeecc32da None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-96d33f84-5189-473d-95a9-81beeecc32da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:28.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:28.558478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:28.559395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:28.559395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:28.559395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:28.559395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:28.559395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 426/837] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:28.564829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab044ee7-8dce-4891-8a85-141cf974dfb5 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:28.638578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:28.686169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab044ee7-8dce-4891-8a85-141cf974dfb5 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:28.733096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fe63b-802e-4efd-b387-ca998d1320c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:28.733748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fe63b-802e-4efd-b387-ca998d1320c7 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:28.734028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fe63b-802e-4efd-b387-ca998d1320c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:28.734292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fe63b-802e-4efd-b387-ca998d1320c7 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:28.735337 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fe63b-802e-4efd-b387-ca998d1320c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:28.735890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fe63b-802e-4efd-b387-ca998d1320c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:28.739216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fe63b-802e-4efd-b387-ca998d1320c7 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:28.760704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 412/838] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:28.769351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d33f84-5189-473d-95a9-81beeecc32da None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:28.823998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:28.883239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d33f84-5189-473d-95a9-81beeecc32da None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:28.920197 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ed23c-5c3a-4644-aa61-7c9d2bd8147e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:28.920969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ed23c-5c3a-4644-aa61-7c9d2bd8147e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:28.921252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ed23c-5c3a-4644-aa61-7c9d2bd8147e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:28.921530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ed23c-5c3a-4644-aa61-7c9d2bd8147e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:28.931478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ed23c-5c3a-4644-aa61-7c9d2bd8147e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:28.932980 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ed23c-5c3a-4644-aa61-7c9d2bd8147e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-421ed23c-5c3a-4644-aa61-7c9d2bd8147e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:28.950119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:28.951196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:28.952145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:28.952145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:28.952145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:28.952145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:28.952145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:28.952145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:28.952145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:28.952145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:28.952145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:40:28.952145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:28.952145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 427/839] 10.222.0.121 () {56 vars in 1122 bytes} [Thu May 26 17:40:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:28.958130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fe63b-802e-4efd-b387-ca998d1320c7 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:29.062654 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fe63b-802e-4efd-b387-ca998d1320c7 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:29.085094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:29.104687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a008eb5-8f50-4673-8390-beb48e651894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:29.104687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a008eb5-8f50-4673-8390-beb48e651894 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:29.104687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a008eb5-8f50-4673-8390-beb48e651894 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:29.104687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a008eb5-8f50-4673-8390-beb48e651894 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:29.106893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a008eb5-8f50-4673-8390-beb48e651894 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:29.106893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a008eb5-8f50-4673-8390-beb48e651894 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:29.107308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a008eb5-8f50-4673-8390-beb48e651894 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:29.127391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d97eb-4fa3-4787-8a92-87373fec419c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:29.127811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d97eb-4fa3-4787-8a92-87373fec419c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:29.127935 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d97eb-4fa3-4787-8a92-87373fec419c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:29.128407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 413/840] 10.222.0.121 () {56 vars in 1154 bytes} [Thu May 26 17:40:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:29.132970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 428/841] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:29 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:29.133912 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a008eb5-8f50-4673-8390-beb48e651894 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:29.135995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5559aa-6dbe-4dff-aa4b-b79a819399e9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:29.136281 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5559aa-6dbe-4dff-aa4b-b79a819399e9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:29.136559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5559aa-6dbe-4dff-aa4b-b79a819399e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:29.156134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2b5559aa-6dbe-4dff-aa4b-b79a819399e9 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:29.157915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2b5559aa-6dbe-4dff-aa4b-b79a819399e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:29.240427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 429/842] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:29 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:29.267023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a008eb5-8f50-4673-8390-beb48e651894 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:29.268402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5559aa-6dbe-4dff-aa4b-b79a819399e9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:29.311123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bade494-b0ae-40a7-9ba6-9485c3d50c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:29.311785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bade494-b0ae-40a7-9ba6-9485c3d50c89 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:29.312070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bade494-b0ae-40a7-9ba6-9485c3d50c89 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:29.312346 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bade494-b0ae-40a7-9ba6-9485c3d50c89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:29.314690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:29.373324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:29.402164 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bade494-b0ae-40a7-9ba6-9485c3d50c89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:29.403676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bade494-b0ae-40a7-9ba6-9485c3d50c89 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:29.404526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5559aa-6dbe-4dff-aa4b-b79a819399e9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:29.460118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e69ef-f6ff-4c72-bad8-e75a9c6c8545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:29.462032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e69ef-f6ff-4c72-bad8-e75a9c6c8545 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:29.462547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e69ef-f6ff-4c72-bad8-e75a9c6c8545 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:29.463276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e69ef-f6ff-4c72-bad8-e75a9c6c8545 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:29.479224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e69ef-f6ff-4c72-bad8-e75a9c6c8545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-af6e69ef-f6ff-4c72-bad8-e75a9c6c8545 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:29.491191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:29.492325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:29.493359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:29.493359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:29.493359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:29.493359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:29.493359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:29.493359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:29.493359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:29.493359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:29.493359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:29.493359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:29.493359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:40:29.493359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:29.499002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 430/843] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:40:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:29.503211 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e69ef-f6ff-4c72-bad8-e75a9c6c8545 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:29.540932 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 414/844] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:29 2022] GET /identity/v3/auth/tokens => generated 1153 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 17:40:29.543802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bade494-b0ae-40a7-9ba6-9485c3d50c89 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:29.615943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:29.622401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e69ef-f6ff-4c72-bad8-e75a9c6c8545 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:29.652720 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:29.666068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950e4af-32c2-46e3-984d-7a2b9dbfea74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:29.667300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950e4af-32c2-46e3-984d-7a2b9dbfea74 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:29.667726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950e4af-32c2-46e3-984d-7a2b9dbfea74 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:29.668155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950e4af-32c2-46e3-984d-7a2b9dbfea74 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:29.669488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950e4af-32c2-46e3-984d-7a2b9dbfea74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:29.670534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950e4af-32c2-46e3-984d-7a2b9dbfea74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:29.673196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950e4af-32c2-46e3-984d-7a2b9dbfea74 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:29.678790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bade494-b0ae-40a7-9ba6-9485c3d50c89 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:29.689310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 431/845] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:40:29 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:29.690997 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950e4af-32c2-46e3-984d-7a2b9dbfea74 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:29.718507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a511eb-8485-4567-916e-4ecf4a00e7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:29.719456 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a511eb-8485-4567-916e-4ecf4a00e7ba None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:29.719736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a511eb-8485-4567-916e-4ecf4a00e7ba None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:29.720008 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a511eb-8485-4567-916e-4ecf4a00e7ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:29.731674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a511eb-8485-4567-916e-4ecf4a00e7ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:29.733216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a511eb-8485-4567-916e-4ecf4a00e7ba None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-c2a511eb-8485-4567-916e-4ecf4a00e7ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:29.757693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:29.763537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:29.764422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:29.764422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:29.764422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:29.764422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:29.764422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 415/846] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:29.764422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a511eb-8485-4567-916e-4ecf4a00e7ba None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:29.825725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950e4af-32c2-46e3-984d-7a2b9dbfea74 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:29.871268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b7843-7409-4402-989e-20722f99051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:29.873157 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b7843-7409-4402-989e-20722f99051f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:29.873823 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b7843-7409-4402-989e-20722f99051f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:29.874344 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b7843-7409-4402-989e-20722f99051f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:29.887246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b7843-7409-4402-989e-20722f99051f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:29.888150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a511eb-8485-4567-916e-4ecf4a00e7ba None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:29.888958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b7843-7409-4402-989e-20722f99051f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-943b7843-7409-4402-989e-20722f99051f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:29.908594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:29.909743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:29.910860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:29.910860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:29.910860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:29.910860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:29.917031 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 432/847] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:29.918485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b7843-7409-4402-989e-20722f99051f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:29.930487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:29.937123 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:29.938421 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637c731-ce9c-49c5-a855-4d3dcb48f90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:29.939208 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637c731-ce9c-49c5-a855-4d3dcb48f90f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:29.939516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637c731-ce9c-49c5-a855-4d3dcb48f90f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:29.942962 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637c731-ce9c-49c5-a855-4d3dcb48f90f None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:29.942962 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7637c731-ce9c-49c5-a855-4d3dcb48f90f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:29.942962 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7637c731-ce9c-49c5-a855-4d3dcb48f90f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:29.943235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7637c731-ce9c-49c5-a855-4d3dcb48f90f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:29.962903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 416/848] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:29.964760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637c731-ce9c-49c5-a855-4d3dcb48f90f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:30.059955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b7843-7409-4402-989e-20722f99051f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:30.094594 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637c731-ce9c-49c5-a855-4d3dcb48f90f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:30.104358 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45927664-6e03-4e63-8227-b915d90e741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:30.105155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45927664-6e03-4e63-8227-b915d90e741e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:30.105442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45927664-6e03-4e63-8227-b915d90e741e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:30.105718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45927664-6e03-4e63-8227-b915d90e741e None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:30.106549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45927664-6e03-4e63-8227-b915d90e741e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:30.106902 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45927664-6e03-4e63-8227-b915d90e741e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:30.118908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45927664-6e03-4e63-8227-b915d90e741e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:30.122801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 433/849] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:30.129567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45927664-6e03-4e63-8227-b915d90e741e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:30.139482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4229853-dfe2-4243-a6a8-5add0f94d03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:30.140317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4229853-dfe2-4243-a6a8-5add0f94d03b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:30.140601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4229853-dfe2-4243-a6a8-5add0f94d03b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:30.141108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4229853-dfe2-4243-a6a8-5add0f94d03b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:30.150702 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4229853-dfe2-4243-a6a8-5add0f94d03b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:30.152438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4229853-dfe2-4243-a6a8-5add0f94d03b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-a4229853-dfe2-4243-a6a8-5add0f94d03b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:30.173159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:30.174188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:30.175112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:30.175112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:30.175112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:30.175112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:30.175112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:30.175112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:30.175112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:30.175112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:30.179066 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 417/850] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:30.180406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4229853-dfe2-4243-a6a8-5add0f94d03b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:30.218333 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:30.251680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45927664-6e03-4e63-8227-b915d90e741e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:30.252638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:30.292066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd8d47-d938-45c4-b673-05f0d45ec494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:30.292873 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd8d47-d938-45c4-b673-05f0d45ec494 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:30.293173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd8d47-d938-45c4-b673-05f0d45ec494 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:30.293424 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd8d47-d938-45c4-b673-05f0d45ec494 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:30.300863 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd8d47-d938-45c4-b673-05f0d45ec494 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:30.302357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd8d47-d938-45c4-b673-05f0d45ec494 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:30.310211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4229853-dfe2-4243-a6a8-5add0f94d03b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-abcd8d47-d938-45c4-b673-05f0d45ec494 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:30.317145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:30.318124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 434/851] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:40:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8a741-df16-42cb-81f5-d66f84d67390 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:30.324151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8a741-df16-42cb-81f5-d66f84d67390 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:30.325005 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8a741-df16-42cb-81f5-d66f84d67390 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:30.325005 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 435/852] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:30.326673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8a741-df16-42cb-81f5-d66f84d67390 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:30.358491 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1625b-add7-4e09-9116-11ac2ffb5d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:30.359874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1625b-add7-4e09-9116-11ac2ffb5d01 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:30.359874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1625b-add7-4e09-9116-11ac2ffb5d01 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:30.360102 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1625b-add7-4e09-9116-11ac2ffb5d01 None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:30.360788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1625b-add7-4e09-9116-11ac2ffb5d01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:30.361175 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1625b-add7-4e09-9116-11ac2ffb5d01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:30.362604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1625b-add7-4e09-9116-11ac2ffb5d01 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:30.375925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 418/853] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:30.380225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b6104-1d17-442a-bb5b-34a2f0b6e5e4 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:30.380586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b6104-1d17-442a-bb5b-34a2f0b6e5e4 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:30.380943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b6104-1d17-442a-bb5b-34a2f0b6e5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:30.398802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-835b6104-1d17-442a-bb5b-34a2f0b6e5e4 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:30.399865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-835b6104-1d17-442a-bb5b-34a2f0b6e5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:30.447840 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8a741-df16-42cb-81f5-d66f84d67390 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:30.474197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 419/854] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:30 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:30.474882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b6104-1d17-442a-bb5b-34a2f0b6e5e4 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:30.490455 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:30.495268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371f638-fe1e-41bd-a11e-e3d62210d7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:30.496234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371f638-fe1e-41bd-a11e-e3d62210d7a4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:30.496657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371f638-fe1e-41bd-a11e-e3d62210d7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:30.497136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371f638-fe1e-41bd-a11e-e3d62210d7a4 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:30.498183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371f638-fe1e-41bd-a11e-e3d62210d7a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:30.498681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371f638-fe1e-41bd-a11e-e3d62210d7a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:30.501156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371f638-fe1e-41bd-a11e-e3d62210d7a4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:30.518614 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:30.525149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 436/855] 10.222.0.121 () {56 vars in 1140 bytes} [Thu May 26 17:40:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:30.526306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371f638-fe1e-41bd-a11e-e3d62210d7a4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:30.584038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b6104-1d17-442a-bb5b-34a2f0b6e5e4 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:30.623848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8149cec-2ec1-4046-bd0d-9836fdc1fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:30.624790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8149cec-2ec1-4046-bd0d-9836fdc1fb14 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:30.625345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8149cec-2ec1-4046-bd0d-9836fdc1fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:30.625627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8149cec-2ec1-4046-bd0d-9836fdc1fb14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:30.626661 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8149cec-2ec1-4046-bd0d-9836fdc1fb14 None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=485c784a043541619ffffc657b21ab97) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:30.627110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8149cec-2ec1-4046-bd0d-9836fdc1fb14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:30.628899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8149cec-2ec1-4046-bd0d-9836fdc1fb14 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:30.634699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371f638-fe1e-41bd-a11e-e3d62210d7a4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:30.635269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 420/856] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:30 2022] GET /identity/v3/role_assignments?user.id=485c784a043541619ffffc657b21ab97&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 610 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:30.635865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8149cec-2ec1-4046-bd0d-9836fdc1fb14 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:30.691151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763c6b9-73ba-4d21-b135-26f4d8ded839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:30.693325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763c6b9-73ba-4d21-b135-26f4d8ded839 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:30.693914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763c6b9-73ba-4d21-b135-26f4d8ded839 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:30.694439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763c6b9-73ba-4d21-b135-26f4d8ded839 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:30.696359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763c6b9-73ba-4d21-b135-26f4d8ded839 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:30.698498 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763c6b9-73ba-4d21-b135-26f4d8ded839 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-2763c6b9-73ba-4d21-b135-26f4d8ded839 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:30.717252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:30.718330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:40:30.719949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:30.728523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 437/857] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:40:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:30.730387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763c6b9-73ba-4d21-b135-26f4d8ded839 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:30.768240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8149cec-2ec1-4046-bd0d-9836fdc1fb14 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:30.771061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:30.809949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:30.814733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d549a5-2f3f-425e-af24-72812b4f310d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:30.815444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d549a5-2f3f-425e-af24-72812b4f310d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:30.815728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d549a5-2f3f-425e-af24-72812b4f310d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:30.816359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d549a5-2f3f-425e-af24-72812b4f310d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:30.877148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763c6b9-73ba-4d21-b135-26f4d8ded839 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:30.899958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d549a5-2f3f-425e-af24-72812b4f310d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:30.902137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d549a5-2f3f-425e-af24-72812b4f310d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:30.921463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a8326-b542-42d2-947b-b81a87ab818a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:30.922596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a8326-b542-42d2-947b-b81a87ab818a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:30.922687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a8326-b542-42d2-947b-b81a87ab818a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:30.927515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a8326-b542-42d2-947b-b81a87ab818a None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:30.927515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a8326-b542-42d2-947b-b81a87ab818a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:30.927515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a8326-b542-42d2-947b-b81a87ab818a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:30.927778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a8326-b542-42d2-947b-b81a87ab818a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:30.933543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 438/858] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:40:30 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:30.934945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104033df-ac57-428b-8e6a-c7a4049c16c2 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:30.935225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104033df-ac57-428b-8e6a-c7a4049c16c2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:30.935501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104033df-ac57-428b-8e6a-c7a4049c16c2 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:30.939021 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 439/859] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:30.942253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104033df-ac57-428b-8e6a-c7a4049c16c2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:31.017049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 421/860] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:30 2022] GET /identity/v3/auth/tokens => generated 1153 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:40:31.018356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae6ec7-23db-42d3-8070-e8ff3a6ffcf0 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:31.018642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae6ec7-23db-42d3-8070-e8ff3a6ffcf0 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:31.018970 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae6ec7-23db-42d3-8070-e8ff3a6ffcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:31.024823 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:31.036406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-aeae6ec7-23db-42d3-8070-e8ff3a6ffcf0 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:31.037595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-aeae6ec7-23db-42d3-8070-e8ff3a6ffcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:31.062670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104033df-ac57-428b-8e6a-c7a4049c16c2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:31.085151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:31.108529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85199340-afe7-4292-8259-077c8f9e21b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:31.108529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85199340-afe7-4292-8259-077c8f9e21b2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:31.108864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85199340-afe7-4292-8259-077c8f9e21b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:31.109054 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85199340-afe7-4292-8259-077c8f9e21b2 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:31.109786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85199340-afe7-4292-8259-077c8f9e21b2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:31.110224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85199340-afe7-4292-8259-077c8f9e21b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:31.114872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85199340-afe7-4292-8259-077c8f9e21b2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:31.115037 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 422/861] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:31 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:31.115037 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae6ec7-23db-42d3-8070-e8ff3a6ffcf0 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:31.123801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 440/862] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:40:30 2022] GET /identity/v3/services?type=volumev3 => generated 126 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:31.134401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85199340-afe7-4292-8259-077c8f9e21b2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:31.256063 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae6ec7-23db-42d3-8070-e8ff3a6ffcf0 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:31.269219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85199340-afe7-4292-8259-077c8f9e21b2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:31.308049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569cdd9-98ac-413c-9607-f161bc8eb39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:31.308619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbcc98-759c-4942-92fa-8b28823b683f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:31.309324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569cdd9-98ac-413c-9607-f161bc8eb39d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:31.309451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbcc98-759c-4942-92fa-8b28823b683f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:31.309701 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbcc98-759c-4942-92fa-8b28823b683f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:31.309972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbcc98-759c-4942-92fa-8b28823b683f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:31.310182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569cdd9-98ac-413c-9607-f161bc8eb39d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:31.310594 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569cdd9-98ac-413c-9607-f161bc8eb39d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:31.310704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbcc98-759c-4942-92fa-8b28823b683f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:31.312026 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbcc98-759c-4942-92fa-8b28823b683f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:31.313419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:31.324485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569cdd9-98ac-413c-9607-f161bc8eb39d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-53fbcc98-759c-4942-92fa-8b28823b683f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:31.325886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:31.328652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:40:31.329979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:31.331077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569cdd9-98ac-413c-9607-f161bc8eb39d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:31.331241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 441/863] 10.222.0.121 () {56 vars in 1137 bytes} [Thu May 26 17:40:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:31.333605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbcc98-759c-4942-92fa-8b28823b683f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-4569cdd9-98ac-413c-9607-f161bc8eb39d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:31.353163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:31.354128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:31.355127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:31.355127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:31.355127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:31.355127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:31.355127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 423/864] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:31.361292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569cdd9-98ac-413c-9607-f161bc8eb39d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:31.391412 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:31.449104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbcc98-759c-4942-92fa-8b28823b683f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:31.481805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569cdd9-98ac-413c-9607-f161bc8eb39d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:31.495352 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5472671-9e17-474f-abfa-f96429332b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:31.496721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5472671-9e17-474f-abfa-f96429332b29 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:31.497227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5472671-9e17-474f-abfa-f96429332b29 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:31.497718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5472671-9e17-474f-abfa-f96429332b29 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:31.498641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5472671-9e17-474f-abfa-f96429332b29 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:31.499178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5472671-9e17-474f-abfa-f96429332b29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:31.501457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5472671-9e17-474f-abfa-f96429332b29 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:31.507106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 442/865] 10.222.0.121 () {56 vars in 1131 bytes} [Thu May 26 17:40:31 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:31.512159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5472671-9e17-474f-abfa-f96429332b29 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:31.524254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4f6c0-f338-4ffe-b4e7-6fb6536b3048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:31.525736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4f6c0-f338-4ffe-b4e7-6fb6536b3048 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:31.526494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4f6c0-f338-4ffe-b4e7-6fb6536b3048 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:31.527031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4f6c0-f338-4ffe-b4e7-6fb6536b3048 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:31.528228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4f6c0-f338-4ffe-b4e7-6fb6536b3048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:31.529217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4f6c0-f338-4ffe-b4e7-6fb6536b3048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:31.531996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4f6c0-f338-4ffe-b4e7-6fb6536b3048 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:31.549659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 424/866] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:31.555500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4f6c0-f338-4ffe-b4e7-6fb6536b3048 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:31.619341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:31.625959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5472671-9e17-474f-abfa-f96429332b29 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:31.635828 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:31.663454 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecb4e7-a15d-4491-b93d-92dfa7fc8741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:31.664349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecb4e7-a15d-4491-b93d-92dfa7fc8741 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:31.665020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecb4e7-a15d-4491-b93d-92dfa7fc8741 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:31.665441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecb4e7-a15d-4491-b93d-92dfa7fc8741 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:31.666452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4f6c0-f338-4ffe-b4e7-6fb6536b3048 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:31.666744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecb4e7-a15d-4491-b93d-92dfa7fc8741 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:31.667222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecb4e7-a15d-4491-b93d-92dfa7fc8741 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:31.668743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecb4e7-a15d-4491-b93d-92dfa7fc8741 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:31.683363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 443/867] 10.222.0.121 () {56 vars in 1131 bytes} [Thu May 26 17:40:31 2022] GET /identity/v3/services?type=compute_legacy => generated 132 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:31.707115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69a46b-ab6b-4777-8f61-a96dfc54ddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:31.707948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69a46b-ab6b-4777-8f61-a96dfc54ddc5 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:31.708231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69a46b-ab6b-4777-8f61-a96dfc54ddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:31.708509 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69a46b-ab6b-4777-8f61-a96dfc54ddc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:31.716572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69a46b-ab6b-4777-8f61-a96dfc54ddc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:31.718049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69a46b-ab6b-4777-8f61-a96dfc54ddc5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-9c69a46b-ab6b-4777-8f61-a96dfc54ddc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:31.734190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:31.736039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:31.738076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:31.738076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:31.738076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:31.738076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:31.738076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:31.738076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:31.738076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:31.738076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:31.738076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 425/868] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:31.742236 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecb4e7-a15d-4491-b93d-92dfa7fc8741 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:31.865465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecb4e7-a15d-4491-b93d-92dfa7fc8741 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:31.904193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931132a-feeb-4776-b90c-0e2837087a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:31.905242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931132a-feeb-4776-b90c-0e2837087a71 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:31.905721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931132a-feeb-4776-b90c-0e2837087a71 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:31.906190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931132a-feeb-4776-b90c-0e2837087a71 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:31.907879 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931132a-feeb-4776-b90c-0e2837087a71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:31.908415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931132a-feeb-4776-b90c-0e2837087a71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:31.910052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931132a-feeb-4776-b90c-0e2837087a71 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:31.928989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 444/869] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:31.930608 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69a46b-ab6b-4777-8f61-a96dfc54ddc5 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:32.042113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69a46b-ab6b-4777-8f61-a96dfc54ddc5 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:32.050613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:32.082094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8aee5-7ca0-44ab-8786-01995afb978f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:32.083151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8aee5-7ca0-44ab-8786-01995afb978f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:32.083636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8aee5-7ca0-44ab-8786-01995afb978f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:32.084106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8aee5-7ca0-44ab-8786-01995afb978f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:32.085152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8aee5-7ca0-44ab-8786-01995afb978f None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=eb9ba9c73956403ba870f6f6b0e687aa) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:32.085739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8aee5-7ca0-44ab-8786-01995afb978f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:32.087464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8aee5-7ca0-44ab-8786-01995afb978f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:32.102997 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 426/870] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:31 2022] GET /identity/v3/role_assignments?user.id=eb9ba9c73956403ba870f6f6b0e687aa&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=4f574961798246a4b12ad2276e67685d => generated 610 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:33.040326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c86e04-bccb-4488-83dd-44b3a7dea66d None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:33.041611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c86e04-bccb-4488-83dd-44b3a7dea66d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:33.042095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c86e04-bccb-4488-83dd-44b3a7dea66d None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:33.044037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 445/871] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:33 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:33.050793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fb107-879f-4c92-9bd8-04243a66864a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:33.051693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fb107-879f-4c92-9bd8-04243a66864a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:33.052189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fb107-879f-4c92-9bd8-04243a66864a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:33.069778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1d2fb107-879f-4c92-9bd8-04243a66864a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:33.070989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1d2fb107-879f-4c92-9bd8-04243a66864a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:33.143821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 427/872] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:33 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:33.167792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c86e04-bccb-4488-83dd-44b3a7dea66d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:33.201227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:33.287390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c86e04-bccb-4488-83dd-44b3a7dea66d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:33.325732 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363098c-9e28-4bb8-8848-81a28d3d230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:33.326939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363098c-9e28-4bb8-8848-81a28d3d230a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:33.327447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363098c-9e28-4bb8-8848-81a28d3d230a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:33.327943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363098c-9e28-4bb8-8848-81a28d3d230a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:33.340205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363098c-9e28-4bb8-8848-81a28d3d230a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-6363098c-9e28-4bb8-8848-81a28d3d230a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:33.349253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:33.350214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:33.351108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:33.351108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:33.351108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:33.351108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:33.351108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:33.351108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:33.351108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:33.351108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:33.351108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:33.351108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:33.351108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:40:33.351108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:33.353888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 446/873] 10.222.0.121 () {56 vars in 1125 bytes} [Thu May 26 17:40:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:33.360580 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fb107-879f-4c92-9bd8-04243a66864a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:33.440519 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b28b1-b38c-4dc8-81e2-7f96a1fc3244 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:33.440846 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b28b1-b38c-4dc8-81e2-7f96a1fc3244 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:33.441068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b28b1-b38c-4dc8-81e2-7f96a1fc3244 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:33.442667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 447/874] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:33.447685 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc4c7b-5e03-4562-baa2-4ffd8b61471b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:33.447967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc4c7b-5e03-4562-baa2-4ffd8b61471b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:33.448253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc4c7b-5e03-4562-baa2-4ffd8b61471b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:33.467358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:33.470878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3dcc4c7b-5e03-4562-baa2-4ffd8b61471b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:33.471191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fb107-879f-4c92-9bd8-04243a66864a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:33.472428 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3dcc4c7b-5e03-4562-baa2-4ffd8b61471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:33.508986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c03cac-698e-4612-8038-cafd4283ca68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:33.509754 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c03cac-698e-4612-8038-cafd4283ca68 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:33.510105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c03cac-698e-4612-8038-cafd4283ca68 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:33.510446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c03cac-698e-4612-8038-cafd4283ca68 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:33.511489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c03cac-698e-4612-8038-cafd4283ca68 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:33.511867 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c03cac-698e-4612-8038-cafd4283ca68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:33.513606 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c03cac-698e-4612-8038-cafd4283ca68 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:33.527921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 428/875] 10.222.0.121 () {56 vars in 1120 bytes} [Thu May 26 17:40:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:33.529481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c03cac-698e-4612-8038-cafd4283ca68 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:33.558981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 448/876] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:33 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:33.581109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc4c7b-5e03-4562-baa2-4ffd8b61471b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:33.613901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:33.654705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c03cac-698e-4612-8038-cafd4283ca68 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:33.702315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c0cdd-9ec0-4458-86c2-345d9265c43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:33.703294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc4c7b-5e03-4562-baa2-4ffd8b61471b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:33.703902 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c0cdd-9ec0-4458-86c2-345d9265c43e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:33.704381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c0cdd-9ec0-4458-86c2-345d9265c43e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:33.705128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c0cdd-9ec0-4458-86c2-345d9265c43e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:33.721315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c0cdd-9ec0-4458-86c2-345d9265c43e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:33.723763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c0cdd-9ec0-4458-86c2-345d9265c43e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:33.740929 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d38d7f-bab2-41fc-a868-a717c823dd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:33.741695 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d38d7f-bab2-41fc-a868-a717c823dd28 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:33.742031 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d38d7f-bab2-41fc-a868-a717c823dd28 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:33.747351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d38d7f-bab2-41fc-a868-a717c823dd28 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:33.747351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d38d7f-bab2-41fc-a868-a717c823dd28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:33.747351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d38d7f-bab2-41fc-a868-a717c823dd28 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-a65c0cdd-9ec0-4458-86c2-345d9265c43e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:33.747585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:33.748383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:33.749271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:33.749271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:33.749271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:33.749271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:33.752418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 429/877] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:33.758021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c0cdd-9ec0-4458-86c2-345d9265c43e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:33.775399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:33.859105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 449/878] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:40:33 2022] POST /identity/v3/services => generated 252 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:40:33.872189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c0cdd-9ec0-4458-86c2-345d9265c43e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:33.925821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa76ce-3a37-4f98-828f-d802acfd4d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:33.926761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa76ce-3a37-4f98-828f-d802acfd4d69 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:33.927234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa76ce-3a37-4f98-828f-d802acfd4d69 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:33.927652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa76ce-3a37-4f98-828f-d802acfd4d69 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:33.928600 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa76ce-3a37-4f98-828f-d802acfd4d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:33.929106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa76ce-3a37-4f98-828f-d802acfd4d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:33.930669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa76ce-3a37-4f98-828f-d802acfd4d69 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:33.945199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 430/879] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:33.950865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d38d7f-bab2-41fc-a868-a717c823dd28 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:34.053595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:34.075561 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d38d7f-bab2-41fc-a868-a717c823dd28 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:34.113226 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b4f55-fc64-49b5-a847-3f94de60fc34 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:34.114095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b4f55-fc64-49b5-a847-3f94de60fc34 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:34.114572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b4f55-fc64-49b5-a847-3f94de60fc34 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:34.116519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 431/880] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:34 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:34.122057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97a46a-645d-41a3-b3d1-8b97fa76a47a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:34.122346 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97a46a-645d-41a3-b3d1-8b97fa76a47a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:34.122629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97a46a-645d-41a3-b3d1-8b97fa76a47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:34.124769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85bbe6-cbf0-4cf3-898c-89b4f645f0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:34.124769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85bbe6-cbf0-4cf3-898c-89b4f645f0ed None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:34.124769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85bbe6-cbf0-4cf3-898c-89b4f645f0ed None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:34.124769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85bbe6-cbf0-4cf3-898c-89b4f645f0ed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:34.135335 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85bbe6-cbf0-4cf3-898c-89b4f645f0ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:34.135981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85bbe6-cbf0-4cf3-898c-89b4f645f0ed None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:34.144024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1a97a46a-645d-41a3-b3d1-8b97fa76a47a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:34.145503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1a97a46a-645d-41a3-b3d1-8b97fa76a47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-9e85bbe6-cbf0-4cf3-898c-89b4f645f0ed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:34.151845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:34.152775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:34.156450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:34.156450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:34.156450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:34.156450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:34.156450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:34.156450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:34.156450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:34.156450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:34.156450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:40:34.156450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:34.156450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 450/881] 10.222.0.121 () {56 vars in 1122 bytes} [Thu May 26 17:40:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:34.160136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85bbe6-cbf0-4cf3-898c-89b4f645f0ed None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:34.240137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:34.248641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 432/882] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:34 2022] POST /identity/v3/auth/tokens => generated 1254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:34.275653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97a46a-645d-41a3-b3d1-8b97fa76a47a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:34.287895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85bbe6-cbf0-4cf3-898c-89b4f645f0ed None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:34.331354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b93b4-567e-4e3b-be86-03ddfb6b3106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:34.332151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b93b4-567e-4e3b-be86-03ddfb6b3106 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:34.332505 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b93b4-567e-4e3b-be86-03ddfb6b3106 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:34.332857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b93b4-567e-4e3b-be86-03ddfb6b3106 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:34.333955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b93b4-567e-4e3b-be86-03ddfb6b3106 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:34.334334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b93b4-567e-4e3b-be86-03ddfb6b3106 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:34.335883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b93b4-567e-4e3b-be86-03ddfb6b3106 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:34.354098 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:34.359367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 451/883] 10.222.0.121 () {56 vars in 1154 bytes} [Thu May 26 17:40:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:34.364900 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b93b4-567e-4e3b-be86-03ddfb6b3106 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:34.402118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97a46a-645d-41a3-b3d1-8b97fa76a47a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:34.442540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ca97f-25d3-4301-8990-9ce05ec3d5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:34.443262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ca97f-25d3-4301-8990-9ce05ec3d5f0 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:34.443548 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ca97f-25d3-4301-8990-9ce05ec3d5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:34.443815 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ca97f-25d3-4301-8990-9ce05ec3d5f0 None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:34.444551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ca97f-25d3-4301-8990-9ce05ec3d5f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:34.445921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ca97f-25d3-4301-8990-9ce05ec3d5f0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:34.487792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b93b4-567e-4e3b-be86-03ddfb6b3106 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:34.523384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2c246-ae18-48d1-b182-29422fcd70c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:34.524027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2c246-ae18-48d1-b182-29422fcd70c4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:34.524304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2c246-ae18-48d1-b182-29422fcd70c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:34.524574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2c246-ae18-48d1-b182-29422fcd70c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:34.555794 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 433/884] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:40:34 2022] POST /identity/v3/services => generated 270 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:40:34.557462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8667b5-9530-4e5f-833a-1c5805c55ab0 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:34.557798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8667b5-9530-4e5f-833a-1c5805c55ab0 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:34.558105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8667b5-9530-4e5f-833a-1c5805c55ab0 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:34.564379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 434/885] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:34 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:34.570479 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea4148-6664-4f49-89e7-0859b2a483da None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:34.570479 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea4148-6664-4f49-89e7-0859b2a483da None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:34.570479 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea4148-6664-4f49-89e7-0859b2a483da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:34.595930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f8ea4148-6664-4f49-89e7-0859b2a483da None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:34.597339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f8ea4148-6664-4f49-89e7-0859b2a483da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:34.631703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:34.636093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2c246-ae18-48d1-b182-29422fcd70c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:34.639618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2c246-ae18-48d1-b182-29422fcd70c4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:34.676948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:34.699015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 435/886] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:34 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:34.722805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea4148-6664-4f49-89e7-0859b2a483da None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:34.808443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 452/887] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:34 2022] GET /identity/v3/auth/tokens => generated 1254 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:40:34.815457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2c246-ae18-48d1-b182-29422fcd70c4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:34.856720 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea4148-6664-4f49-89e7-0859b2a483da None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:34.899123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d309efc-c1ed-405a-a4e6-54b97c8ee5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:34.899966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d309efc-c1ed-405a-a4e6-54b97c8ee5a4 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:34.900253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d309efc-c1ed-405a-a4e6-54b97c8ee5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:34.900526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d309efc-c1ed-405a-a4e6-54b97c8ee5a4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:34.901188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d309efc-c1ed-405a-a4e6-54b97c8ee5a4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:34.902561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d309efc-c1ed-405a-a4e6-54b97c8ee5a4 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-0d309efc-c1ed-405a-a4e6-54b97c8ee5a4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:34.916689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:34.917739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:34.918872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 436/888] 10.222.0.121 () {56 vars in 1131 bytes} [Thu May 26 17:40:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:34.925214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d309efc-c1ed-405a-a4e6-54b97c8ee5a4 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:34.942854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:34.946440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2c246-ae18-48d1-b182-29422fcd70c4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:34.980741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:34.985820 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fedd8-0493-4aae-a4da-190e6b8a9d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:34.986781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fedd8-0493-4aae-a4da-190e6b8a9d87 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:34.987125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fedd8-0493-4aae-a4da-190e6b8a9d87 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:34.987402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fedd8-0493-4aae-a4da-190e6b8a9d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:34.999342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fedd8-0493-4aae-a4da-190e6b8a9d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:35.001525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fedd8-0493-4aae-a4da-190e6b8a9d87 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-373fedd8-0493-4aae-a4da-190e6b8a9d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:35.020703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:35.022799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:35.023785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:35.023785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:35.023785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:35.023785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:35.023785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 453/889] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:35.028415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fedd8-0493-4aae-a4da-190e6b8a9d87 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:35.034154 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d309efc-c1ed-405a-a4e6-54b97c8ee5a4 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:35.074416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525dbf79-665c-4b7d-8efa-522c56dd002e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:35.075738 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dbf79-665c-4b7d-8efa-522c56dd002e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:35.076120 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dbf79-665c-4b7d-8efa-522c56dd002e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:35.077016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dbf79-665c-4b7d-8efa-522c56dd002e None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:35.078232 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525dbf79-665c-4b7d-8efa-522c56dd002e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:35.078645 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525dbf79-665c-4b7d-8efa-522c56dd002e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:35.080912 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525dbf79-665c-4b7d-8efa-522c56dd002e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:35.087828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 437/890] 10.222.0.121 () {56 vars in 1127 bytes} [Thu May 26 17:40:34 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:35.094764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525dbf79-665c-4b7d-8efa-522c56dd002e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:35.155082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fedd8-0493-4aae-a4da-190e6b8a9d87 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:35.196837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36382684-6c25-4fe7-83e2-9b3abcc36a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:35.198048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36382684-6c25-4fe7-83e2-9b3abcc36a16 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:35.198529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36382684-6c25-4fe7-83e2-9b3abcc36a16 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:35.199078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36382684-6c25-4fe7-83e2-9b3abcc36a16 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:35.200006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36382684-6c25-4fe7-83e2-9b3abcc36a16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:35.200526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36382684-6c25-4fe7-83e2-9b3abcc36a16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:35.202350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36382684-6c25-4fe7-83e2-9b3abcc36a16 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:35.206128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525dbf79-665c-4b7d-8efa-522c56dd002e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:35.216922 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 454/891] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:35.225370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36382684-6c25-4fe7-83e2-9b3abcc36a16 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:35.225549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:35.238567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:35.246459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d211969-7c2c-4a04-88dd-de2f4f73e38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:35.247370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d211969-7c2c-4a04-88dd-de2f4f73e38c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:35.247490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d211969-7c2c-4a04-88dd-de2f4f73e38c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:35.247703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d211969-7c2c-4a04-88dd-de2f4f73e38c None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:35.248412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d211969-7c2c-4a04-88dd-de2f4f73e38c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:35.248748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d211969-7c2c-4a04-88dd-de2f4f73e38c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:35.250943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d211969-7c2c-4a04-88dd-de2f4f73e38c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:35.262993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 438/892] 10.222.0.121 () {56 vars in 1127 bytes} [Thu May 26 17:40:35 2022] GET /identity/v3/services?type=object-store => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:35.341849 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36382684-6c25-4fe7-83e2-9b3abcc36a16 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:35.381494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af393c5d-4b61-4c3f-a95f-9c0191b4497a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:35.382644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af393c5d-4b61-4c3f-a95f-9c0191b4497a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:35.383210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af393c5d-4b61-4c3f-a95f-9c0191b4497a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:35.383870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af393c5d-4b61-4c3f-a95f-9c0191b4497a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:35.393906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af393c5d-4b61-4c3f-a95f-9c0191b4497a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:35.395689 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af393c5d-4b61-4c3f-a95f-9c0191b4497a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-af393c5d-4b61-4c3f-a95f-9c0191b4497a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:35.411618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:35.412713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:35.413803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:35.413803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:35.413803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:35.413803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:35.413803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:35.413803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:35.413803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:35.413803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:35.416269 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 455/893] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:35.422487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d211969-7c2c-4a04-88dd-de2f4f73e38c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:35.535202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d211969-7c2c-4a04-88dd-de2f4f73e38c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:35.571843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7ad7b-aa8b-46f4-8c67-590fef53b867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:35.572847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ad7b-aa8b-46f4-8c67-590fef53b867 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:35.573278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ad7b-aa8b-46f4-8c67-590fef53b867 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:35.573701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ad7b-aa8b-46f4-8c67-590fef53b867 None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:35.574629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7ad7b-aa8b-46f4-8c67-590fef53b867 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:35.575125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7ad7b-aa8b-46f4-8c67-590fef53b867 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:35.576722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7ad7b-aa8b-46f4-8c67-590fef53b867 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:35.591305 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 439/894] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:35.597978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af393c5d-4b61-4c3f-a95f-9c0191b4497a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:35.677963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:35.710553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af393c5d-4b61-4c3f-a95f-9c0191b4497a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:35.747990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7915a-83f2-478e-a868-3a27b615b3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:35.755314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7915a-83f2-478e-a868-3a27b615b3db None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:35.755314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7915a-83f2-478e-a868-3a27b615b3db None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:35.755314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7915a-83f2-478e-a868-3a27b615b3db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:35.755314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7915a-83f2-478e-a868-3a27b615b3db None admin] RBAC: Adding query filter params (role.id=1808269227c74f569faa53261e45ca40, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=485c784a043541619ffffc657b21ab97) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:35.755314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7915a-83f2-478e-a868-3a27b615b3db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:35.755314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7915a-83f2-478e-a868-3a27b615b3db None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:35.760310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 456/895] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:35 2022] GET /identity/v3/role_assignments?user.id=485c784a043541619ffffc657b21ab97&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=1808269227c74f569faa53261e45ca40 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:36.271103 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82572ee3-dc83-4464-9f0b-b3558c2594fc None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:36.271612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82572ee3-dc83-4464-9f0b-b3558c2594fc None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:36.271726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82572ee3-dc83-4464-9f0b-b3558c2594fc None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:36.273329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 440/896] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:36.280843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2791b-9f86-46ac-b194-d12d4da2fe05 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:36.281544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2791b-9f86-46ac-b194-d12d4da2fe05 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:36.281544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2791b-9f86-46ac-b194-d12d4da2fe05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:36.300178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-47f2791b-9f86-46ac-b194-d12d4da2fe05 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:36.301429 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-47f2791b-9f86-46ac-b194-d12d4da2fe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:36.383061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 457/897] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:36 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:36.409464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82572ee3-dc83-4464-9f0b-b3558c2594fc None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:36.457417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:36.543381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82572ee3-dc83-4464-9f0b-b3558c2594fc None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:36.594861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac54b8e-8697-4f66-9c86-24969efc7858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:36.595913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac54b8e-8697-4f66-9c86-24969efc7858 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:36.596196 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac54b8e-8697-4f66-9c86-24969efc7858 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:36.596468 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac54b8e-8697-4f66-9c86-24969efc7858 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:36.597133 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac54b8e-8697-4f66-9c86-24969efc7858 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:36.598514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac54b8e-8697-4f66-9c86-24969efc7858 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-bac54b8e-8697-4f66-9c86-24969efc7858 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:36.618109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:36.619189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:40:36.620026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:36.623356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 441/898] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:40:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:36.626334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2791b-9f86-46ac-b194-d12d4da2fe05 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:36.700951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:36.742255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2791b-9f86-46ac-b194-d12d4da2fe05 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:36.780701 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d2d20-1e8e-41db-bcbe-2b10d2b28a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:36.782389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d2d20-1e8e-41db-bcbe-2b10d2b28a6c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:36.782726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d2d20-1e8e-41db-bcbe-2b10d2b28a6c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:36.783126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d2d20-1e8e-41db-bcbe-2b10d2b28a6c None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:36.798700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d2d20-1e8e-41db-bcbe-2b10d2b28a6c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:36.798700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d2d20-1e8e-41db-bcbe-2b10d2b28a6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:36.798700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d2d20-1e8e-41db-bcbe-2b10d2b28a6c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:36.798700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 458/899] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:40:36 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:36.799052 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac54b8e-8697-4f66-9c86-24969efc7858 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:36.918312 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac54b8e-8697-4f66-9c86-24969efc7858 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:36.920286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:36.959504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1153ef-3287-44f8-a283-1e829943b753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:36.960134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1153ef-3287-44f8-a283-1e829943b753 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:36.960423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1153ef-3287-44f8-a283-1e829943b753 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:36.960697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1153ef-3287-44f8-a283-1e829943b753 None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:36.961410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1153ef-3287-44f8-a283-1e829943b753 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:36.961712 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1153ef-3287-44f8-a283-1e829943b753 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:36.963108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1153ef-3287-44f8-a283-1e829943b753 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:36.969126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 442/900] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:40:36 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:36.974797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d2d20-1e8e-41db-bcbe-2b10d2b28a6c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:37.099290 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d2d20-1e8e-41db-bcbe-2b10d2b28a6c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:37.132208 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02191cc-b16e-4967-8da4-4bca3b218962 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:37.132525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02191cc-b16e-4967-8da4-4bca3b218962 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:37.132759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02191cc-b16e-4967-8da4-4bca3b218962 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:37.138489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 443/901] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:37.139065 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:37.143278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07742992-b7e3-4940-866a-a3e511d3249f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:37.143707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01c1b9-fb51-40ec-8afd-dc2e0a62a89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:37.144204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07742992-b7e3-4940-866a-a3e511d3249f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:37.144409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01c1b9-fb51-40ec-8afd-dc2e0a62a89d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:37.144650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01c1b9-fb51-40ec-8afd-dc2e0a62a89d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:37.144916 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01c1b9-fb51-40ec-8afd-dc2e0a62a89d None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:37.145343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07742992-b7e3-4940-866a-a3e511d3249f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:37.145713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01c1b9-fb51-40ec-8afd-dc2e0a62a89d None admin] RBAC: Adding query filter params (interface=public, service_id=00d03439c086420d89b92fde221e0440, region_id=RegionOne) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:37.146021 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01c1b9-fb51-40ec-8afd-dc2e0a62a89d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:37.147505 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01c1b9-fb51-40ec-8afd-dc2e0a62a89d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:37.154864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 459/902] 10.222.0.121 () {56 vars in 1256 bytes} [Thu May 26 17:40:36 2022] GET /identity/v3/endpoints?service_id=00d03439c086420d89b92fde221e0440&interface=public®ion_id=RegionOne => generated 195 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:37.160824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01c1b9-fb51-40ec-8afd-dc2e0a62a89d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:37.169368 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-07742992-b7e3-4940-866a-a3e511d3249f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:37.171437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-07742992-b7e3-4940-866a-a3e511d3249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:37.266189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 444/903] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:37 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:37.292824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01c1b9-fb51-40ec-8afd-dc2e0a62a89d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:37.293416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07742992-b7e3-4940-866a-a3e511d3249f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:37.342357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b3459-4784-4c8e-ba9f-3bc9d5ec6acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:37.343354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b3459-4784-4c8e-ba9f-3bc9d5ec6acb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:37.344393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b3459-4784-4c8e-ba9f-3bc9d5ec6acb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:37.344942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b3459-4784-4c8e-ba9f-3bc9d5ec6acb None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:37.345784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b3459-4784-4c8e-ba9f-3bc9d5ec6acb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:37.347480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b3459-4784-4c8e-ba9f-3bc9d5ec6acb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:37.354596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 460/904] 10.222.0.121 () {56 vars in 1092 bytes} [Thu May 26 17:40:37 2022] GET /identity/v3/services => generated 1274 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:40:37.409890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:37.416511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07742992-b7e3-4940-866a-a3e511d3249f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:37.458147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8487a6-6f45-42b5-b25a-6a2a6e02024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:37.459029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8487a6-6f45-42b5-b25a-6a2a6e02024c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:37.459312 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8487a6-6f45-42b5-b25a-6a2a6e02024c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:37.459587 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8487a6-6f45-42b5-b25a-6a2a6e02024c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:37.460259 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8487a6-6f45-42b5-b25a-6a2a6e02024c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:37.461659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8487a6-6f45-42b5-b25a-6a2a6e02024c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-fd8487a6-6f45-42b5-b25a-6a2a6e02024c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:37.479731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:37.480728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:37.481582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 445/905] 10.222.0.121 () {56 vars in 1137 bytes} [Thu May 26 17:40:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:37.488310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b3459-4784-4c8e-ba9f-3bc9d5ec6acb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:37.567472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:37.616500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b3459-4784-4c8e-ba9f-3bc9d5ec6acb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:37.655894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4dfcf-7337-4f13-bfaf-ac3a718f2d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:37.656911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4dfcf-7337-4f13-bfaf-ac3a718f2d8d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:37.657192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4dfcf-7337-4f13-bfaf-ac3a718f2d8d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:37.657468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4dfcf-7337-4f13-bfaf-ac3a718f2d8d None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:37.658181 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4dfcf-7337-4f13-bfaf-ac3a718f2d8d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:37.658483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4dfcf-7337-4f13-bfaf-ac3a718f2d8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:37.659907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4dfcf-7337-4f13-bfaf-ac3a718f2d8d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:37.665208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 461/906] 10.222.0.121 () {56 vars in 1131 bytes} [Thu May 26 17:40:37 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:37.674212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8487a6-6f45-42b5-b25a-6a2a6e02024c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:37.746855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a2aeb-dbd1-4e8b-8ef3-41c5078b9a74 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:37.751743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a2aeb-dbd1-4e8b-8ef3-41c5078b9a74 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:37.751743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a2aeb-dbd1-4e8b-8ef3-41c5078b9a74 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:37.751743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 462/907] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:37.754040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba766c-4b93-4e3f-8d4a-66e9c4a5342d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:37.754346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba766c-4b93-4e3f-8d4a-66e9c4a5342d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:37.754595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba766c-4b93-4e3f-8d4a-66e9c4a5342d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:37.775118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-15ba766c-4b93-4e3f-8d4a-66e9c4a5342d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:37.776311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-15ba766c-4b93-4e3f-8d4a-66e9c4a5342d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:37.803662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8487a6-6f45-42b5-b25a-6a2a6e02024c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:37.842594 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ac798-29aa-45ed-8a0f-df4d8f9fb096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:37.843798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ac798-29aa-45ed-8a0f-df4d8f9fb096 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:37.844329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ac798-29aa-45ed-8a0f-df4d8f9fb096 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:37.844865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ac798-29aa-45ed-8a0f-df4d8f9fb096 None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:37.845783 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ac798-29aa-45ed-8a0f-df4d8f9fb096 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:37.846388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ac798-29aa-45ed-8a0f-df4d8f9fb096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:37.848092 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ac798-29aa-45ed-8a0f-df4d8f9fb096 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:37.854702 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 446/908] 10.222.0.121 () {56 vars in 1131 bytes} [Thu May 26 17:40:37 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:37.861366 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ac798-29aa-45ed-8a0f-df4d8f9fb096 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:37.867796 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 463/909] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:37 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:37.891880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba766c-4b93-4e3f-8d4a-66e9c4a5342d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:37.902748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:37.963825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:37.984994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ac798-29aa-45ed-8a0f-df4d8f9fb096 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:38.019061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba766c-4b93-4e3f-8d4a-66e9c4a5342d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:38.021523 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8754a4-e5da-445c-8d94-e319de03945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:38.022297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8754a4-e5da-445c-8d94-e319de03945b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:38.022680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8754a4-e5da-445c-8d94-e319de03945b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:38.023100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8754a4-e5da-445c-8d94-e319de03945b None admin] PATH_INFO: `/v3/endpoints` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:38.023997 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8754a4-e5da-445c-8d94-e319de03945b None admin] RBAC: Adding query filter params (interface=public, service_id=d708a7c0e3324507b440803b9864623d, region_id=RegionOne) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:38.024428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8754a4-e5da-445c-8d94-e319de03945b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:38.025887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8754a4-e5da-445c-8d94-e319de03945b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:38.034704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 447/910] 10.222.0.121 () {56 vars in 1256 bytes} [Thu May 26 17:40:37 2022] GET /identity/v3/endpoints?service_id=d708a7c0e3324507b440803b9864623d&interface=public®ion_id=RegionOne => generated 195 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:38.040082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8754a4-e5da-445c-8d94-e319de03945b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:38.060709 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb88955-75ae-4fbf-aecd-6de1be3760b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:38.061478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb88955-75ae-4fbf-aecd-6de1be3760b9 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:38.061762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb88955-75ae-4fbf-aecd-6de1be3760b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:38.062034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb88955-75ae-4fbf-aecd-6de1be3760b9 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:38.063339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb88955-75ae-4fbf-aecd-6de1be3760b9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:38.064866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb88955-75ae-4fbf-aecd-6de1be3760b9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:38.132990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 464/911] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:40:37 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:40:38.134695 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc048bc-f7ee-48d2-ad36-9228c7f0acd9 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:38.135218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc048bc-f7ee-48d2-ad36-9228c7f0acd9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:38.135756 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc048bc-f7ee-48d2-ad36-9228c7f0acd9 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:38.137528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 465/912] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:38.143592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4c3df-dd0d-43bf-bda8-274a65ff11ae None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:38.144216 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4c3df-dd0d-43bf-bda8-274a65ff11ae None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:38.144697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4c3df-dd0d-43bf-bda8-274a65ff11ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:38.153978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:38.163705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f3a4c3df-dd0d-43bf-bda8-274a65ff11ae None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:38.165469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f3a4c3df-dd0d-43bf-bda8-274a65ff11ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:38.167769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8754a4-e5da-445c-8d94-e319de03945b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:38.222966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711aa86-206a-4388-9d9f-ef6caf000ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:38.223684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711aa86-206a-4388-9d9f-ef6caf000ce2 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:38.223969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711aa86-206a-4388-9d9f-ef6caf000ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:38.224246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711aa86-206a-4388-9d9f-ef6caf000ce2 None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:38.224942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711aa86-206a-4388-9d9f-ef6caf000ce2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:38.226326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711aa86-206a-4388-9d9f-ef6caf000ce2 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:38.243316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 448/913] 10.222.0.121 () {56 vars in 1092 bytes} [Thu May 26 17:40:38 2022] GET /identity/v3/services => generated 1504 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:40:38.263893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 466/914] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:38 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:38.288054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711aa86-206a-4388-9d9f-ef6caf000ce2 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:38.411977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711aa86-206a-4388-9d9f-ef6caf000ce2 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:38.453424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d710b1-ea13-443e-9b34-a1fc422dd359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:38.454603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d710b1-ea13-443e-9b34-a1fc422dd359 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:38.454917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d710b1-ea13-443e-9b34-a1fc422dd359 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:38.455196 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d710b1-ea13-443e-9b34-a1fc422dd359 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:38.467514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d710b1-ea13-443e-9b34-a1fc422dd359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-46d710b1-ea13-443e-9b34-a1fc422dd359 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:38.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:38.477278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:38.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 449/915] 10.222.0.121 () {56 vars in 1125 bytes} [Thu May 26 17:40:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:38.483596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4c3df-dd0d-43bf-bda8-274a65ff11ae None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:38.532221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:38.599651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4c3df-dd0d-43bf-bda8-274a65ff11ae None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:38.637078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fd8de-5c26-4b9e-8b9d-6400ad05955f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:38.637747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fd8de-5c26-4b9e-8b9d-6400ad05955f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:38.638027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fd8de-5c26-4b9e-8b9d-6400ad05955f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:38.638354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fd8de-5c26-4b9e-8b9d-6400ad05955f None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:38.639125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fd8de-5c26-4b9e-8b9d-6400ad05955f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:38.639475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fd8de-5c26-4b9e-8b9d-6400ad05955f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:38.641490 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fd8de-5c26-4b9e-8b9d-6400ad05955f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:38.650651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 467/916] 10.222.0.121 () {56 vars in 1120 bytes} [Thu May 26 17:40:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:38.656787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d710b1-ea13-443e-9b34-a1fc422dd359 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:38.669377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:38.766285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d710b1-ea13-443e-9b34-a1fc422dd359 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:38.802326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad231b4-1c78-4796-8bac-846716d9ba4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:38.803571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad231b4-1c78-4796-8bac-846716d9ba4e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:38.804061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad231b4-1c78-4796-8bac-846716d9ba4e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:38.804536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad231b4-1c78-4796-8bac-846716d9ba4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:38.816331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad231b4-1c78-4796-8bac-846716d9ba4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:38.818162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad231b4-1c78-4796-8bac-846716d9ba4e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-6ad231b4-1c78-4796-8bac-846716d9ba4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:38.836311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:38.837233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:38.838122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:38.838122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:38.838122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:38.838122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:38.840369 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 450/917] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:38.846894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fd8de-5c26-4b9e-8b9d-6400ad05955f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:38.953242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fd8de-5c26-4b9e-8b9d-6400ad05955f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:38.955547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:38.987959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c1538-178a-4cdd-875c-64b71df5f7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:38.988719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c1538-178a-4cdd-875c-64b71df5f7fc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:38.988976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c1538-178a-4cdd-875c-64b71df5f7fc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:38.993299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c1538-178a-4cdd-875c-64b71df5f7fc None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:38.993299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c1538-178a-4cdd-875c-64b71df5f7fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:38.993299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c1538-178a-4cdd-875c-64b71df5f7fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:38.993299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c1538-178a-4cdd-875c-64b71df5f7fc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:39.005160 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 468/918] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:39.010361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad231b4-1c78-4796-8bac-846716d9ba4e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:39.074538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:39.118234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad231b4-1c78-4796-8bac-846716d9ba4e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:39.153834 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765fa2b-43c5-45cf-80f2-621bf95a2cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:39.154658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765fa2b-43c5-45cf-80f2-621bf95a2cd2 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:39.154969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765fa2b-43c5-45cf-80f2-621bf95a2cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:39.155245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765fa2b-43c5-45cf-80f2-621bf95a2cd2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:39.163870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765fa2b-43c5-45cf-80f2-621bf95a2cd2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:39.165415 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765fa2b-43c5-45cf-80f2-621bf95a2cd2 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-7765fa2b-43c5-45cf-80f2-621bf95a2cd2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:39.181324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:39.182315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:39.183298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:39.183298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:39.183298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:39.183298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:39.183298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:39.183298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:39.183298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:39.183298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:39.183298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:40:39.183298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:39.186984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 451/919] 10.222.0.121 () {56 vars in 1122 bytes} [Thu May 26 17:40:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:39.189304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c1538-178a-4cdd-875c-64b71df5f7fc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:39.292968 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c1538-178a-4cdd-875c-64b71df5f7fc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:39.329554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f016-2cda-4974-8e8e-871565bf136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:39.330240 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f016-2cda-4974-8e8e-871565bf136c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:39.330525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f016-2cda-4974-8e8e-871565bf136c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:39.330800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f016-2cda-4974-8e8e-871565bf136c None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:39.331660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81f016-2cda-4974-8e8e-871565bf136c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:39.331970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81f016-2cda-4974-8e8e-871565bf136c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:39.333472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81f016-2cda-4974-8e8e-871565bf136c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:39.352418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 469/920] 10.222.0.121 () {56 vars in 1154 bytes} [Thu May 26 17:40:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:39.357996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765fa2b-43c5-45cf-80f2-621bf95a2cd2 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:39.473575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765fa2b-43c5-45cf-80f2-621bf95a2cd2 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:39.495239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:39.509775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2eec8-f6e9-40c5-a6fd-9f17fd4cb014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:39.510632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2eec8-f6e9-40c5-a6fd-9f17fd4cb014 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:39.510992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2eec8-f6e9-40c5-a6fd-9f17fd4cb014 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:39.511273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2eec8-f6e9-40c5-a6fd-9f17fd4cb014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:39.523429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2eec8-f6e9-40c5-a6fd-9f17fd4cb014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:39.524890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2eec8-f6e9-40c5-a6fd-9f17fd4cb014 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-f0b2eec8-f6e9-40c5-a6fd-9f17fd4cb014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:39.543544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:39.544489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:39.545389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:39.545389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:39.545389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:39.545389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:39.545389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 452/921] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:39.551452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f016-2cda-4974-8e8e-871565bf136c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:39.579785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:39.671530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f016-2cda-4974-8e8e-871565bf136c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:39.708020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efcd5a-f702-4bd2-9084-07fdc2ca9a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:39.709115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efcd5a-f702-4bd2-9084-07fdc2ca9a54 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:39.709606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efcd5a-f702-4bd2-9084-07fdc2ca9a54 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:39.710086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efcd5a-f702-4bd2-9084-07fdc2ca9a54 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:39.711106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efcd5a-f702-4bd2-9084-07fdc2ca9a54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:39.711613 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efcd5a-f702-4bd2-9084-07fdc2ca9a54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:39.713183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efcd5a-f702-4bd2-9084-07fdc2ca9a54 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:39.726326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 470/922] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:39.731773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2eec8-f6e9-40c5-a6fd-9f17fd4cb014 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:39.774174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aae9f-23a9-409e-9497-5575f3fc5889 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:39.774792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aae9f-23a9-409e-9497-5575f3fc5889 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:39.775440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aae9f-23a9-409e-9497-5575f3fc5889 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:39.777315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 471/923] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:39 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:39.784083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b4ffd-4a6e-4fea-b717-0ba227171f20 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:39.784734 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b4ffd-4a6e-4fea-b717-0ba227171f20 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:39.785214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b4ffd-4a6e-4fea-b717-0ba227171f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:39.804156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9c7b4ffd-4a6e-4fea-b717-0ba227171f20 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:39.805742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9c7b4ffd-4a6e-4fea-b717-0ba227171f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:39.842017 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2eec8-f6e9-40c5-a6fd-9f17fd4cb014 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:39.878372 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452b6db-ef73-4d45-8d51-73aa1798252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:39.879219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452b6db-ef73-4d45-8d51-73aa1798252e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:39.879625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452b6db-ef73-4d45-8d51-73aa1798252e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:39.879989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452b6db-ef73-4d45-8d51-73aa1798252e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:39.890918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452b6db-ef73-4d45-8d51-73aa1798252e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:39.890918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452b6db-ef73-4d45-8d51-73aa1798252e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:39.892261 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 472/924] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:39 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-9452b6db-ef73-4d45-8d51-73aa1798252e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:39.905035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:39.906540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:39.907653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:39.907653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:39.907653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:39.907653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:39.907653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:39.907653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:39.907653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:39.907653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:39.907653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 453/925] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:39.912811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b4ffd-4a6e-4fea-b717-0ba227171f20 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:39.913059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452b6db-ef73-4d45-8d51-73aa1798252e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:39.929032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:39.946437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:40.021997 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452b6db-ef73-4d45-8d51-73aa1798252e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:40.033888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b4ffd-4a6e-4fea-b717-0ba227171f20 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:40.058685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef0bc2-8b9b-49b6-8dc8-887605a4cabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:40.059313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef0bc2-8b9b-49b6-8dc8-887605a4cabb None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:40.059596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef0bc2-8b9b-49b6-8dc8-887605a4cabb None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:40.059872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef0bc2-8b9b-49b6-8dc8-887605a4cabb None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:40.060660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef0bc2-8b9b-49b6-8dc8-887605a4cabb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:40.060972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef0bc2-8b9b-49b6-8dc8-887605a4cabb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:40.062556 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef0bc2-8b9b-49b6-8dc8-887605a4cabb None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:40.080586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 454/926] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:40.081518 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef0bc2-8b9b-49b6-8dc8-887605a4cabb None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:40.082413 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb616f-13da-4a5b-9255-df4c17342274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:40.083337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb616f-13da-4a5b-9255-df4c17342274 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:40.083687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb616f-13da-4a5b-9255-df4c17342274 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:40.084027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb616f-13da-4a5b-9255-df4c17342274 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:40.084921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb616f-13da-4a5b-9255-df4c17342274 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:40.087263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb616f-13da-4a5b-9255-df4c17342274 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f2cb616f-13da-4a5b-9255-df4c17342274 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:40.102142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:40.103597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:40.104753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 473/927] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:40:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:40.115897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb616f-13da-4a5b-9255-df4c17342274 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:40.207148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:40.210707 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:40.223198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef0bc2-8b9b-49b6-8dc8-887605a4cabb None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:40.230602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb616f-13da-4a5b-9255-df4c17342274 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:40.266972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1ecf0-1018-4c7d-ab94-af073d5a5652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:40.267938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa1a94-4e8b-43de-a22d-a5ef68d856ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:40.268518 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1ecf0-1018-4c7d-ab94-af073d5a5652 None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:40.268626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa1a94-4e8b-43de-a22d-a5ef68d856ba None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:40.268864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa1a94-4e8b-43de-a22d-a5ef68d856ba None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:40.269134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa1a94-4e8b-43de-a22d-a5ef68d856ba None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:40.269381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1ecf0-1018-4c7d-ab94-af073d5a5652 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:40.269809 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1ecf0-1018-4c7d-ab94-af073d5a5652 None admin] PATH_INFO: `/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/485c784a043541619ffffc657b21ab97/roles/1808269227c74f569faa53261e45ca40` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:40.269909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa1a94-4e8b-43de-a22d-a5ef68d856ba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:40.270193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa1a94-4e8b-43de-a22d-a5ef68d856ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:40.271648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa1a94-4e8b-43de-a22d-a5ef68d856ba None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:40.272791 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1ecf0-1018-4c7d-ab94-af073d5a5652 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61dea65b34b47a186151c6a63bc7d79, user_id=485c784a043541619ffffc657b21ab97, role_id=1808269227c74f569faa53261e45ca40)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:40.274679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1ecf0-1018-4c7d-ab94-af073d5a5652 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:40.282092 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 474/928] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:40:40 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:40.284337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa1a94-4e8b-43de-a22d-a5ef68d856ba None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:40.360295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 455/929] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:40:40 2022] PUT /identity/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/485c784a043541619ffffc657b21ab97/roles/1808269227c74f569faa53261e45ca40 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:40:40.412996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa1a94-4e8b-43de-a22d-a5ef68d856ba None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:40.458336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbda999-6398-43c0-aafa-cb5f921c078d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:40.459111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbda999-6398-43c0-aafa-cb5f921c078d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:40.459398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbda999-6398-43c0-aafa-cb5f921c078d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:40.459719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbda999-6398-43c0-aafa-cb5f921c078d None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:40.460547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbda999-6398-43c0-aafa-cb5f921c078d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:40.460856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbda999-6398-43c0-aafa-cb5f921c078d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:40.462289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbda999-6398-43c0-aafa-cb5f921c078d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:40.470395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 475/930] 10.222.0.121 () {56 vars in 1119 bytes} [Thu May 26 17:40:40 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:40.476999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1ecf0-1018-4c7d-ab94-af073d5a5652 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:40.592251 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1ecf0-1018-4c7d-ab94-af073d5a5652 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:40.599214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f73d-d500-4935-a3e1-eb1bb347de02 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:40.600163 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f73d-d500-4935-a3e1-eb1bb347de02 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:40.600441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f73d-d500-4935-a3e1-eb1bb347de02 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:40.602559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 476/931] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:40 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:40.610213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c7571-bb0e-4494-87ca-891876b9464b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:40.610548 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c7571-bb0e-4494-87ca-891876b9464b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:40.610766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c7571-bb0e-4494-87ca-891876b9464b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:40.617348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:40.633425 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e83c7571-bb0e-4494-87ca-891876b9464b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:40.635302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e83c7571-bb0e-4494-87ca-891876b9464b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:40.640520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc643ef-6c08-4815-ab7d-ed9feda0b6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:40.640520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc643ef-6c08-4815-ab7d-ed9feda0b6a6 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:40.640520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc643ef-6c08-4815-ab7d-ed9feda0b6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:40.640520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc643ef-6c08-4815-ab7d-ed9feda0b6a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:40.640923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc643ef-6c08-4815-ab7d-ed9feda0b6a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:40.640923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc643ef-6c08-4815-ab7d-ed9feda0b6a6 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:40.713424 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 477/932] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:40 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:40.715090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249bc1e-637f-461f-b1d6-94f4ab833c44 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:40.715541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249bc1e-637f-461f-b1d6-94f4ab833c44 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:40.715959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249bc1e-637f-461f-b1d6-94f4ab833c44 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:40.717379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 478/933] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:40.723604 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32690a-373a-40bd-8cca-b7bdbb4607a8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:40.724076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32690a-373a-40bd-8cca-b7bdbb4607a8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:40.724501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32690a-373a-40bd-8cca-b7bdbb4607a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:40.725438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 456/934] 10.222.0.121 () {60 vars in 1149 bytes} [Thu May 26 17:40:40 2022] POST /identity/v3/endpoints => generated 348 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:40:40.738110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc643ef-6c08-4815-ab7d-ed9feda0b6a6 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:40.750484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9e32690a-373a-40bd-8cca-b7bdbb4607a8 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:40.751915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9e32690a-373a-40bd-8cca-b7bdbb4607a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:40.775228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:40.850924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 479/935] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:40 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:40.870712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32690a-373a-40bd-8cca-b7bdbb4607a8 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:40.889850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc643ef-6c08-4815-ab7d-ed9feda0b6a6 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:40.931454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba512a1-9d84-4f87-9107-f8a83d147387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:40.932398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba512a1-9d84-4f87-9107-f8a83d147387 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:40.932641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba512a1-9d84-4f87-9107-f8a83d147387 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:40.932912 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba512a1-9d84-4f87-9107-f8a83d147387 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:40.933561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba512a1-9d84-4f87-9107-f8a83d147387 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:40.936073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba512a1-9d84-4f87-9107-f8a83d147387 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:40.941728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-aba512a1-9d84-4f87-9107-f8a83d147387 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:40.950217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:40.958669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:40.959545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 457/936] 10.222.0.121 () {56 vars in 1137 bytes} [Thu May 26 17:40:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:40.960376 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba512a1-9d84-4f87-9107-f8a83d147387 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:40.993688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32690a-373a-40bd-8cca-b7bdbb4607a8 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:41.036993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a38c61-e686-45ae-acad-9fc83a75e8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:41.037915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a38c61-e686-45ae-acad-9fc83a75e8d0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:41.038204 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a38c61-e686-45ae-acad-9fc83a75e8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:41.038480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a38c61-e686-45ae-acad-9fc83a75e8d0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:41.039330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a38c61-e686-45ae-acad-9fc83a75e8d0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:41.040889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a38c61-e686-45ae-acad-9fc83a75e8d0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:41.048982 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e8a38c61-e686-45ae-acad-9fc83a75e8d0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:41.063662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:41.065053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:41.066094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:41.068729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 480/937] 10.222.0.121 () {56 vars in 1131 bytes} [Thu May 26 17:40:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:41.075158 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a38c61-e686-45ae-acad-9fc83a75e8d0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:41.084509 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba512a1-9d84-4f87-9107-f8a83d147387 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:41.131785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22681ba4-614b-455b-b41e-9c422918934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:41.132413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22681ba4-614b-455b-b41e-9c422918934f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:41.132692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22681ba4-614b-455b-b41e-9c422918934f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:41.132963 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22681ba4-614b-455b-b41e-9c422918934f None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:41.133673 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22681ba4-614b-455b-b41e-9c422918934f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:41.133977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22681ba4-614b-455b-b41e-9c422918934f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:41.135573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22681ba4-614b-455b-b41e-9c422918934f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:41.145895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 458/938] 10.222.0.121 () {56 vars in 1131 bytes} [Thu May 26 17:40:40 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:41.147577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22681ba4-614b-455b-b41e-9c422918934f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:41.202628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a38c61-e686-45ae-acad-9fc83a75e8d0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:41.233211 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:41.252440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5afe49-6273-461e-976e-92951985f353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:41.253962 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5afe49-6273-461e-976e-92951985f353 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:41.254981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5afe49-6273-461e-976e-92951985f353 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:41.255493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5afe49-6273-461e-976e-92951985f353 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:41.256730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5afe49-6273-461e-976e-92951985f353 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:41.257225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5afe49-6273-461e-976e-92951985f353 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:41.259602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5afe49-6273-461e-976e-92951985f353 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:41.272146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 481/939] 10.222.0.121 () {56 vars in 1127 bytes} [Thu May 26 17:40:41 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:41.273193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5afe49-6273-461e-976e-92951985f353 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:41.286547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22681ba4-614b-455b-b41e-9c422918934f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:41.333461 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66b8d-cb10-422c-be3e-fa93edf1f59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:41.334678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66b8d-cb10-422c-be3e-fa93edf1f59c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:41.335428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66b8d-cb10-422c-be3e-fa93edf1f59c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:41.335888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66b8d-cb10-422c-be3e-fa93edf1f59c None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:41.337177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e66b8d-cb10-422c-be3e-fa93edf1f59c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:41.337668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e66b8d-cb10-422c-be3e-fa93edf1f59c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:41.339805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e66b8d-cb10-422c-be3e-fa93edf1f59c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:41.343111 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:41.352804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 459/940] 10.222.0.121 () {56 vars in 1131 bytes} [Thu May 26 17:40:41 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:41.354118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66b8d-cb10-422c-be3e-fa93edf1f59c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:41.402140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5afe49-6273-461e-976e-92951985f353 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:41.440535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdfa86-a1e8-453d-80ce-62a060ea50b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:41.441296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdfa86-a1e8-453d-80ce-62a060ea50b7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:41.441582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdfa86-a1e8-453d-80ce-62a060ea50b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:41.441852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdfa86-a1e8-453d-80ce-62a060ea50b7 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:41.442807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdfa86-a1e8-453d-80ce-62a060ea50b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:41.443018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdfa86-a1e8-453d-80ce-62a060ea50b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:41.444333 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdfa86-a1e8-453d-80ce-62a060ea50b7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:41.455048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 482/941] 10.222.0.121 () {56 vars in 1127 bytes} [Thu May 26 17:40:41 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:41.456390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdfa86-a1e8-453d-80ce-62a060ea50b7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:41.492007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66b8d-cb10-422c-be3e-fa93edf1f59c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:41.516480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:41.537061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73062d-2e3c-494f-bb7c-77959926ead1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:41.538079 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73062d-2e3c-494f-bb7c-77959926ead1 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:41.538638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73062d-2e3c-494f-bb7c-77959926ead1 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:41.539114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73062d-2e3c-494f-bb7c-77959926ead1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:41.540044 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73062d-2e3c-494f-bb7c-77959926ead1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:41.542367 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73062d-2e3c-494f-bb7c-77959926ead1 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:41.571514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdfa86-a1e8-453d-80ce-62a060ea50b7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:41.609519 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85db4f0-2db6-4f9f-932d-4d1cf041c893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:41.610177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85db4f0-2db6-4f9f-932d-4d1cf041c893 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:41.610558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85db4f0-2db6-4f9f-932d-4d1cf041c893 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:41.610877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85db4f0-2db6-4f9f-932d-4d1cf041c893 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:41.611714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85db4f0-2db6-4f9f-932d-4d1cf041c893 None admin] RBAC: Adding query filter params (interface=public, service_id=e0123a373d7a443f8a9800e7c1c7cdf6, region_id=RegionOne) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:41.612024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85db4f0-2db6-4f9f-932d-4d1cf041c893 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:41.613386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85db4f0-2db6-4f9f-932d-4d1cf041c893 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:41.619529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 483/942] 10.222.0.121 () {56 vars in 1256 bytes} [Thu May 26 17:40:41 2022] GET /identity/v3/endpoints?service_id=e0123a373d7a443f8a9800e7c1c7cdf6&interface=public®ion_id=RegionOne => generated 195 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:41.624778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85db4f0-2db6-4f9f-932d-4d1cf041c893 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:41.644343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 460/943] 10.222.0.121 () {60 vars in 1149 bytes} [Thu May 26 17:40:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:40:41.754785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85db4f0-2db6-4f9f-932d-4d1cf041c893 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:41.775738 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:41.803699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c180b7-1d92-4728-ac48-bf9a19233d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:41.804484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c180b7-1d92-4728-ac48-bf9a19233d68 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:41.804839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c180b7-1d92-4728-ac48-bf9a19233d68 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:41.805188 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c180b7-1d92-4728-ac48-bf9a19233d68 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:41.805953 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c180b7-1d92-4728-ac48-bf9a19233d68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:41.807589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c180b7-1d92-4728-ac48-bf9a19233d68 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:41.815303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 484/944] 10.222.0.121 () {56 vars in 1092 bytes} [Thu May 26 17:40:41 2022] GET /identity/v3/services => generated 1504 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:40:42.716248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f2f40-5e52-42c6-8ae5-267cc20b5d94 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:42.717133 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f2f40-5e52-42c6-8ae5-267cc20b5d94 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:42.717600 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f2f40-5e52-42c6-8ae5-267cc20b5d94 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:42.720483 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 461/945] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:42 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:42.726263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104d296-1d62-412e-9af6-fb9908705198 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:42.727137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104d296-1d62-412e-9af6-fb9908705198 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:42.727635 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104d296-1d62-412e-9af6-fb9908705198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:42.746248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1104d296-1d62-412e-9af6-fb9908705198 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:42.747474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1104d296-1d62-412e-9af6-fb9908705198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:42.820199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 485/946] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:42 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:42.844390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f2f40-5e52-42c6-8ae5-267cc20b5d94 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:42.912038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:42.961696 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f2f40-5e52-42c6-8ae5-267cc20b5d94 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:43.000733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67fd0c-cb39-42f1-8b25-4fe1d31c757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:43.001484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67fd0c-cb39-42f1-8b25-4fe1d31c757d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:43.001989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67fd0c-cb39-42f1-8b25-4fe1d31c757d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:43.002261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67fd0c-cb39-42f1-8b25-4fe1d31c757d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:43.013401 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67fd0c-cb39-42f1-8b25-4fe1d31c757d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-8b67fd0c-cb39-42f1-8b25-4fe1d31c757d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:43.021792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:43.022726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:43.023746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 462/947] 10.222.0.121 () {56 vars in 1125 bytes} [Thu May 26 17:40:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:43.031535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104d296-1d62-412e-9af6-fb9908705198 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:43.104380 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bb4a5-86a8-4538-bcb4-f7400cfe8023 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:43.104699 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bb4a5-86a8-4538-bcb4-f7400cfe8023 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:43.104922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bb4a5-86a8-4538-bcb4-f7400cfe8023 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:43.106441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 463/948] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:43.113551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac91fa-03ed-48ad-8d41-1e904f1b66d6 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:43.113841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac91fa-03ed-48ad-8d41-1e904f1b66d6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:43.114104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac91fa-03ed-48ad-8d41-1e904f1b66d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:43.135761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-e3ac91fa-03ed-48ad-8d41-1e904f1b66d6 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:43.137095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e3ac91fa-03ed-48ad-8d41-1e904f1b66d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:43.152080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104d296-1d62-412e-9af6-fb9908705198 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:43.161791 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:43.189398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25f7c8-9fd3-4135-b2f1-95ef8425bb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:43.190492 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25f7c8-9fd3-4135-b2f1-95ef8425bb93 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:43.191045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25f7c8-9fd3-4135-b2f1-95ef8425bb93 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:43.191515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25f7c8-9fd3-4135-b2f1-95ef8425bb93 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:43.192449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25f7c8-9fd3-4135-b2f1-95ef8425bb93 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:43.192947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25f7c8-9fd3-4135-b2f1-95ef8425bb93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:43.194690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25f7c8-9fd3-4135-b2f1-95ef8425bb93 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:43.205108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 486/949] 10.222.0.121 () {56 vars in 1120 bytes} [Thu May 26 17:40:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:43.210361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25f7c8-9fd3-4135-b2f1-95ef8425bb93 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:43.227677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 464/950] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:43 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:43.250005 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac91fa-03ed-48ad-8d41-1e904f1b66d6 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:43.305276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:43.352048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25f7c8-9fd3-4135-b2f1-95ef8425bb93 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:43.364530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac91fa-03ed-48ad-8d41-1e904f1b66d6 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:43.400424 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9dc047-b463-41ac-a772-ee4b5a7129ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:43.401225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dc047-b463-41ac-a772-ee4b5a7129ce None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:43.401501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dc047-b463-41ac-a772-ee4b5a7129ce None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:43.401769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dc047-b463-41ac-a772-ee4b5a7129ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:43.404460 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c2ed1-bcbb-465e-9f3c-a2a225eb6c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:43.405133 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c2ed1-bcbb-465e-9f3c-a2a225eb6c90 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:43.405410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c2ed1-bcbb-465e-9f3c-a2a225eb6c90 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:43.406195 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c2ed1-bcbb-465e-9f3c-a2a225eb6c90 None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:43.407562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c2ed1-bcbb-465e-9f3c-a2a225eb6c90 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:43.409023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c2ed1-bcbb-465e-9f3c-a2a225eb6c90 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:43.410394 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: WARNING keystone.server.flask.common [None req-0e3c2ed1-bcbb-465e-9f3c-a2a225eb6c90 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 17:40:43.417427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9dc047-b463-41ac-a772-ee4b5a7129ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:43.419197 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9dc047-b463-41ac-a772-ee4b5a7129ce None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-8e9dc047-b463-41ac-a772-ee4b5a7129ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:43.440979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:43.441958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:43.442889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:43.442889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:43.442889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:43.442889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:43.442889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 487/951] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:43.450431 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9dc047-b463-41ac-a772-ee4b5a7129ce None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:43.469154 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.sql.core [None req-0e3c2ed1-bcbb-465e-9f3c-a2a225eb6c90 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 17:40:43.469154 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 17:40:43.469154 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [parameters: {'id': '086f2729023b4038bcec4476e6bbd459', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 17:40:43.469154 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92082) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 17:40:43.473448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-0e3c2ed1-bcbb-465e-9f3c-a2a225eb6c90 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 086f2729023b4038bcec4476e6bbd459.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 086f2729023b4038bcec4476e6bbd459. May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application result = self._query(query) May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application conn.query(q) May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 17:40:43.481062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application result.read() May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 26 17:40:43.488252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return project_ref.to_dict() May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application next(self.gen) May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application yield resource May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application next(self.gen) May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self.session.rollback() May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self.force_reraise() May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise self.value May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application session.commit() May 26 17:40:43.489204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._prepare_impl() May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self.session.flush() May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._flush(objects) May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application compat.raise_( May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application flush_context.execute() May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 17:40:43.490098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rec.execute(self) May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 17:40:43.491377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application result = self._query(query) May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application conn.query(q) May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application result.read() May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 17:40:43.492677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [parameters: {'id': '086f2729023b4038bcec4476e6bbd459', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:43.494539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:43.496200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:43.498383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:43.498383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 26 17:40:43.498383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 26 17:40:43.498383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:43.498383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:43.498383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 26 17:40:43.498383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.Conflict( May 26 17:40:43.498383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 086f2729023b4038bcec4476e6bbd459. May 26 17:40:43.498383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:43.498383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 465/952] 10.222.0.121 () {60 vars in 1145 bytes} [Thu May 26 17:40:43 2022] POST /identity/v3/projects => generated 262 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:43.498383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c2ed1-bcbb-465e-9f3c-a2a225eb6c90 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:43.594733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9dc047-b463-41ac-a772-ee4b5a7129ce None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:43.635416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c2ed1-bcbb-465e-9f3c-a2a225eb6c90 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:43.637377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:43.637493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f16f75-c72c-4510-a9fe-4a58abb5f4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:43.638395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f16f75-c72c-4510-a9fe-4a58abb5f4fa None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:43.638753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f16f75-c72c-4510-a9fe-4a58abb5f4fa None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:43.639140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f16f75-c72c-4510-a9fe-4a58abb5f4fa None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:43.641812 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f16f75-c72c-4510-a9fe-4a58abb5f4fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:43.642459 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f16f75-c72c-4510-a9fe-4a58abb5f4fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:43.644060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f16f75-c72c-4510-a9fe-4a58abb5f4fa None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:43.665420 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 488/953] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:43.667398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f16f75-c72c-4510-a9fe-4a58abb5f4fa None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:43.684871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682e6e2-fb40-4a8f-b4c8-bc69777dbc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:43.685780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682e6e2-fb40-4a8f-b4c8-bc69777dbc94 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:43.686119 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682e6e2-fb40-4a8f-b4c8-bc69777dbc94 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:43.686398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682e6e2-fb40-4a8f-b4c8-bc69777dbc94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:43.695239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682e6e2-fb40-4a8f-b4c8-bc69777dbc94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:43.700383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682e6e2-fb40-4a8f-b4c8-bc69777dbc94 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-0682e6e2-fb40-4a8f-b4c8-bc69777dbc94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:43.713940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:43.714947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:43.717416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:43.717416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:43.717416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:43.717416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:43.717416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:43.717416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:43.717416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:43.717416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:43.720939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 466/954] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:43.721937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682e6e2-fb40-4a8f-b4c8-bc69777dbc94 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:43.767421 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:43.787956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f16f75-c72c-4510-a9fe-4a58abb5f4fa None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:43.829539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400a07-5aa8-4d6f-a434-e283807df71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:43.830801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400a07-5aa8-4d6f-a434-e283807df71f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:43.831359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400a07-5aa8-4d6f-a434-e283807df71f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:43.831858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400a07-5aa8-4d6f-a434-e283807df71f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:43.840280 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09400a07-5aa8-4d6f-a434-e283807df71f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:43.842087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09400a07-5aa8-4d6f-a434-e283807df71f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:43.849054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682e6e2-fb40-4a8f-b4c8-bc69777dbc94 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-09400a07-5aa8-4d6f-a434-e283807df71f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:43.860560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:43.861574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:43.862536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:43.862536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:43.862536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:43.862536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:43.862536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:43.862536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:43.862536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:43.862536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:43.862536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:40:43.862536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:43.869678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 489/955] 10.222.0.121 () {56 vars in 1122 bytes} [Thu May 26 17:40:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:43.871480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400a07-5aa8-4d6f-a434-e283807df71f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:43.886489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56bfc6-58c7-449f-957a-d2a436bf3c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:43.887184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56bfc6-58c7-449f-957a-d2a436bf3c47 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:43.887496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56bfc6-58c7-449f-957a-d2a436bf3c47 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:43.887859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56bfc6-58c7-449f-957a-d2a436bf3c47 None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:43.888621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56bfc6-58c7-449f-957a-d2a436bf3c47 None admin] RBAC: Adding query filter params (name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:43.888926 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56bfc6-58c7-449f-957a-d2a436bf3c47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:43.891379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56bfc6-58c7-449f-957a-d2a436bf3c47 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:43.914955 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 467/956] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:43 2022] GET /identity/v3/projects?name=service => generated 413 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:44.001527 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400a07-5aa8-4d6f-a434-e283807df71f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:44.010801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf9544-fc55-41c5-9110-7000ab3d0600 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:44.011117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf9544-fc55-41c5-9110-7000ab3d0600 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:44.011391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf9544-fc55-41c5-9110-7000ab3d0600 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:44.013134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 468/957] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:44.019055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36aadb6-8126-4b8d-9e4d-693a2401da8b None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:44.019362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36aadb6-8126-4b8d-9e4d-693a2401da8b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:44.019614 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36aadb6-8126-4b8d-9e4d-693a2401da8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:44.040740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c36aadb6-8126-4b8d-9e4d-693a2401da8b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:44.042280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c36aadb6-8126-4b8d-9e4d-693a2401da8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:44.051119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aa98c-5da2-4fb2-b09c-67944eee8597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:44.053003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aa98c-5da2-4fb2-b09c-67944eee8597 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:44.053535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aa98c-5da2-4fb2-b09c-67944eee8597 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:44.054045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aa98c-5da2-4fb2-b09c-67944eee8597 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:44.055646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aa98c-5da2-4fb2-b09c-67944eee8597 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:44.056540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aa98c-5da2-4fb2-b09c-67944eee8597 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:44.058546 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aa98c-5da2-4fb2-b09c-67944eee8597 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:44.063495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:44.074505 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:44.088646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 490/958] 10.222.0.121 () {56 vars in 1154 bytes} [Thu May 26 17:40:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:44.098085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aa98c-5da2-4fb2-b09c-67944eee8597 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:44.143604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 469/959] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:44 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:44.166289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36aadb6-8126-4b8d-9e4d-693a2401da8b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:44.219505 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aa98c-5da2-4fb2-b09c-67944eee8597 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:44.258162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac266b-127e-45ab-85a7-422f2a8bb879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:44.258850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac266b-127e-45ab-85a7-422f2a8bb879 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:44.259133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac266b-127e-45ab-85a7-422f2a8bb879 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:44.259409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac266b-127e-45ab-85a7-422f2a8bb879 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:44.281729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36aadb6-8126-4b8d-9e4d-693a2401da8b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:44.323534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b6d41-4e48-4913-a7dd-60993f891463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:44.324342 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b6d41-4e48-4913-a7dd-60993f891463 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:44.324622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b6d41-4e48-4913-a7dd-60993f891463 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:44.324899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b6d41-4e48-4913-a7dd-60993f891463 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:44.325662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b6d41-4e48-4913-a7dd-60993f891463 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:44.327182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b6d41-4e48-4913-a7dd-60993f891463 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:44.336665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:44.340180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac266b-127e-45ab-85a7-422f2a8bb879 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:44.341690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac266b-127e-45ab-85a7-422f2a8bb879 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-766b6d41-4e48-4913-a7dd-60993f891463 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:44.341793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:44.342764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:44.354274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 470/960] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:44.355138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53726dd2-8fd9-40df-844a-3aba9bc80f04 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:44.355138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53726dd2-8fd9-40df-844a-3aba9bc80f04 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:44.355138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53726dd2-8fd9-40df-844a-3aba9bc80f04 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:44.355138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 471/961] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:44.355138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53726dd2-8fd9-40df-844a-3aba9bc80f04 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:44.371011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:44.453571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 491/962] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:44 2022] GET /identity/v3/auth/tokens => generated 1466 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:40:44.455270 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2631a6f-052f-4a0e-9617-aafcb2b21964 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:44.455674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2631a6f-052f-4a0e-9617-aafcb2b21964 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:44.456079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2631a6f-052f-4a0e-9617-aafcb2b21964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:44.471760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53726dd2-8fd9-40df-844a-3aba9bc80f04 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:44.473309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c2631a6f-052f-4a0e-9617-aafcb2b21964 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:44.474348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c2631a6f-052f-4a0e-9617-aafcb2b21964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:44.511276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0b7b4-5f42-4c92-9a8d-d849711966d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:44.511951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0b7b4-5f42-4c92-9a8d-d849711966d9 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:44.512232 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0b7b4-5f42-4c92-9a8d-d849711966d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:44.512506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0b7b4-5f42-4c92-9a8d-d849711966d9 None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:44.513215 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0b7b4-5f42-4c92-9a8d-d849711966d9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:44.513520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0b7b4-5f42-4c92-9a8d-d849711966d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:44.515025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0b7b4-5f42-4c92-9a8d-d849711966d9 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:44.520530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 472/963] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:44 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:44.521418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0b7b4-5f42-4c92-9a8d-d849711966d9 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:44.554713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 492/964] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:44 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:44.555658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2631a6f-052f-4a0e-9617-aafcb2b21964 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:44.587670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:44.630939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:44.653934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0b7b4-5f42-4c92-9a8d-d849711966d9 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:44.667066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2631a6f-052f-4a0e-9617-aafcb2b21964 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:44.690686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419d500-1537-4564-864c-f3c53cd9b238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:44.691524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419d500-1537-4564-864c-f3c53cd9b238 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:44.691802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419d500-1537-4564-864c-f3c53cd9b238 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:44.692072 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419d500-1537-4564-864c-f3c53cd9b238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:44.701842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934acaa0-1790-4024-a27b-a78a0d80e26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:44.703064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934acaa0-1790-4024-a27b-a78a0d80e26b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:44.703226 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419d500-1537-4564-864c-f3c53cd9b238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:44.703390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934acaa0-1790-4024-a27b-a78a0d80e26b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:44.703711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934acaa0-1790-4024-a27b-a78a0d80e26b None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:44.704296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934acaa0-1790-4024-a27b-a78a0d80e26b None admin] RBAC: Adding query filter params (type=compute) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:44.704511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419d500-1537-4564-864c-f3c53cd9b238 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:44.704612 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934acaa0-1790-4024-a27b-a78a0d80e26b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:44.705953 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934acaa0-1790-4024-a27b-a78a0d80e26b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:44.711064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 493/965] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:44 2022] GET /identity/v3/services?type=compute => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:44.711670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934acaa0-1790-4024-a27b-a78a0d80e26b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d419d500-1537-4564-864c-f3c53cd9b238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:44.730142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:44.731775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:44.732686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:44.732686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:44.732686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:44.732686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:44.735707 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 473/966] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:44.741245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419d500-1537-4564-864c-f3c53cd9b238 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:44.854330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:44.868949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934acaa0-1790-4024-a27b-a78a0d80e26b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:44.886541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419d500-1537-4564-864c-f3c53cd9b238 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:44.911400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacacf4-156f-4beb-bb41-232c6f00aab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:44.912555 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacacf4-156f-4beb-bb41-232c6f00aab7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:44.912837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacacf4-156f-4beb-bb41-232c6f00aab7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:44.913210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacacf4-156f-4beb-bb41-232c6f00aab7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:44.913876 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacacf4-156f-4beb-bb41-232c6f00aab7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:44.915299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacacf4-156f-4beb-bb41-232c6f00aab7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:44.923226 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:44.928245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2b96b-f1ad-4f9e-a745-a3b1cb1b6797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:44.929003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2b96b-f1ad-4f9e-a745-a3b1cb1b6797 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:44.929283 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2b96b-f1ad-4f9e-a745-a3b1cb1b6797 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:44.929565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2b96b-f1ad-4f9e-a745-a3b1cb1b6797 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:44.930275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2b96b-f1ad-4f9e-a745-a3b1cb1b6797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:44.930575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2b96b-f1ad-4f9e-a745-a3b1cb1b6797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-3aacacf4-156f-4beb-bb41-232c6f00aab7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:44.930686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:44.931677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:44.932535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 494/967] 10.222.0.121 () {56 vars in 1131 bytes} [Thu May 26 17:40:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:44.935069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2b96b-f1ad-4f9e-a745-a3b1cb1b6797 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:44.939824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacacf4-156f-4beb-bb41-232c6f00aab7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:44.949438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 474/968] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:44.956901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2b96b-f1ad-4f9e-a745-a3b1cb1b6797 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:45.073985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacacf4-156f-4beb-bb41-232c6f00aab7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:45.086585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2b96b-f1ad-4f9e-a745-a3b1cb1b6797 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:45.122535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2922c-95fb-4e43-b6c7-511e1cc890bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:45.123548 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2922c-95fb-4e43-b6c7-511e1cc890bb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:45.124734 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2922c-95fb-4e43-b6c7-511e1cc890bb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:45.125212 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2922c-95fb-4e43-b6c7-511e1cc890bb None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:45.126146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2922c-95fb-4e43-b6c7-511e1cc890bb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:45.126667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2922c-95fb-4e43-b6c7-511e1cc890bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:45.128273 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2922c-95fb-4e43-b6c7-511e1cc890bb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:45.133555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b198c95-50bf-47e2-affe-fe9966aac5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:45.134316 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 495/969] 10.222.0.121 () {56 vars in 1127 bytes} [Thu May 26 17:40:44 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:45.134443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b198c95-50bf-47e2-affe-fe9966aac5bc None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:45.134708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b198c95-50bf-47e2-affe-fe9966aac5bc None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:45.135196 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b198c95-50bf-47e2-affe-fe9966aac5bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:45.139681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2922c-95fb-4e43-b6c7-511e1cc890bb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:45.143356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b198c95-50bf-47e2-affe-fe9966aac5bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:45.145999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b198c95-50bf-47e2-affe-fe9966aac5bc None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-0b198c95-50bf-47e2-affe-fe9966aac5bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:45.165803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:45.166808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:45.168014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:45.168014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:45.168014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:45.168014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:45.168014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:45.168014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:45.168014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:45.168014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:45.170400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:45.172996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 475/970] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:45.174686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b198c95-50bf-47e2-affe-fe9966aac5bc None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:45.246559 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:45.271012 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2922c-95fb-4e43-b6c7-511e1cc890bb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:45.308697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab0f2b-13e1-4d8c-be4d-e615d50797b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:45.309733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab0f2b-13e1-4d8c-be4d-e615d50797b7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:45.310236 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab0f2b-13e1-4d8c-be4d-e615d50797b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:45.310718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab0f2b-13e1-4d8c-be4d-e615d50797b7 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:45.311907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab0f2b-13e1-4d8c-be4d-e615d50797b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:45.313078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab0f2b-13e1-4d8c-be4d-e615d50797b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:45.314784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab0f2b-13e1-4d8c-be4d-e615d50797b7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:45.320269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b198c95-50bf-47e2-affe-fe9966aac5bc None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:45.320934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 496/971] 10.222.0.121 () {56 vars in 1127 bytes} [Thu May 26 17:40:45 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:45.326122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab0f2b-13e1-4d8c-be4d-e615d50797b7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:45.373585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3db4b-2ae3-4d62-8cbe-48c6b9b7fcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:45.375455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3db4b-2ae3-4d62-8cbe-48c6b9b7fcf9 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:45.375945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3db4b-2ae3-4d62-8cbe-48c6b9b7fcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:45.376590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3db4b-2ae3-4d62-8cbe-48c6b9b7fcf9 None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:45.378071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3db4b-2ae3-4d62-8cbe-48c6b9b7fcf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:45.378980 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3db4b-2ae3-4d62-8cbe-48c6b9b7fcf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:45.381045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3db4b-2ae3-4d62-8cbe-48c6b9b7fcf9 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:45.398104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 476/972] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:45.403658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3db4b-2ae3-4d62-8cbe-48c6b9b7fcf9 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:45.429633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:45.448129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab0f2b-13e1-4d8c-be4d-e615d50797b7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:45.488416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9795b-6cda-4991-84c1-579bab43b40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:45.489090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9795b-6cda-4991-84c1-579bab43b40a None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:45.489375 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9795b-6cda-4991-84c1-579bab43b40a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:45.489677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9795b-6cda-4991-84c1-579bab43b40a None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:45.490531 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9795b-6cda-4991-84c1-579bab43b40a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:45.492125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9795b-6cda-4991-84c1-579bab43b40a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:45.534879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3db4b-2ae3-4d62-8cbe-48c6b9b7fcf9 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:45.551292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:45.571153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 497/973] 10.222.0.121 () {60 vars in 1149 bytes} [Thu May 26 17:40:45 2022] POST /identity/v3/endpoints => generated 351 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:40:45.585748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab2f32-9605-4c82-ae2b-14d9a575ff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:45.586621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab2f32-9605-4c82-ae2b-14d9a575ff30 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:45.587157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab2f32-9605-4c82-ae2b-14d9a575ff30 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:45.587545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab2f32-9605-4c82-ae2b-14d9a575ff30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:45.588532 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab2f32-9605-4c82-ae2b-14d9a575ff30 None admin] RBAC: Adding query filter params (role.id=1808269227c74f569faa53261e45ca40, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=485c784a043541619ffffc657b21ab97) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:45.589074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab2f32-9605-4c82-ae2b-14d9a575ff30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:45.590837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab2f32-9605-4c82-ae2b-14d9a575ff30 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:45.607047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 477/974] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:45 2022] GET /identity/v3/role_assignments?user.id=485c784a043541619ffffc657b21ab97&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=1808269227c74f569faa53261e45ca40 => generated 610 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:46.301106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e64b91-5990-4613-8883-7678392fa63f None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:46.301629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e64b91-5990-4613-8883-7678392fa63f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:46.301629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e64b91-5990-4613-8883-7678392fa63f None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:46.303262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 498/975] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:46.308552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122e26a-bb3c-4669-bbc4-59bce9bb29fe None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:46.308946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122e26a-bb3c-4669-bbc4-59bce9bb29fe None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:46.309127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122e26a-bb3c-4669-bbc4-59bce9bb29fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:46.326422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1122e26a-bb3c-4669-bbc4-59bce9bb29fe None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:46.327996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1122e26a-bb3c-4669-bbc4-59bce9bb29fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:46.410327 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 478/976] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:46 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:46.437393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e64b91-5990-4613-8883-7678392fa63f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:46.553317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e64b91-5990-4613-8883-7678392fa63f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:46.589307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0063caa-4085-4e47-8d4d-cfd9ecf241b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:46.589985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0063caa-4085-4e47-8d4d-cfd9ecf241b7 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:46.590263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0063caa-4085-4e47-8d4d-cfd9ecf241b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:46.590535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0063caa-4085-4e47-8d4d-cfd9ecf241b7 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:46.591266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0063caa-4085-4e47-8d4d-cfd9ecf241b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:46.592706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0063caa-4085-4e47-8d4d-cfd9ecf241b7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:46.594983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: WARNING keystone.server.flask.common [None req-c0063caa-4085-4e47-8d4d-cfd9ecf241b7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 17:40:46.640139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.sql.core [None req-c0063caa-4085-4e47-8d4d-cfd9ecf241b7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 17:40:46.640139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 17:40:46.640139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [parameters: {'user_id': '36e229f6914a4936a516c5a61710ca6d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 17:40:46.640139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92083) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-c0063caa-4085-4e47-8d4d-cfd9ecf241b7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application result = self._query(query) May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application conn.query(q) May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application result.read() May 26 17:40:46.658251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 17:40:46.660446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application next(self.gen) May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application yield resource May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application next(self.gen) May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self.session.rollback() May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self.force_reraise() May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise self.value May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application session.commit() May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 17:40:46.662720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._prepare_impl() May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self.session.flush() May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._flush(objects) May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application compat.raise_( May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application flush_context.execute() May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rec.execute(self) May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 17:40:46.664862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application result = connection._execute_20( May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 17:40:46.666220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application result = self._query(query) May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application conn.query(q) May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application result.read() May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 17:40:46.667276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [parameters: {'user_id': '36e229f6914a4936a516c5a61710ca6d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:46.668234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 26 17:40:46.670104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 26 17:40:46.673272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 26 17:40:46.673272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 26 17:40:46.673272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 17:40:46.673272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 17:40:46.673272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 17:40:46.673272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:46.673272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 499/977] 10.222.0.121 () {60 vars in 1136 bytes} [Thu May 26 17:40:46 2022] POST /identity/v3/users => generated 160 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:46.674189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122e26a-bb3c-4669-bbc4-59bce9bb29fe None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:46.775735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122e26a-bb3c-4669-bbc4-59bce9bb29fe None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:46.777266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:46.814257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389b7df-b547-4339-8b14-01d884f97529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:46.815093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389b7df-b547-4339-8b14-01d884f97529 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:46.815375 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389b7df-b547-4339-8b14-01d884f97529 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:46.815674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389b7df-b547-4339-8b14-01d884f97529 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:46.823815 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389b7df-b547-4339-8b14-01d884f97529 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:46.825282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389b7df-b547-4339-8b14-01d884f97529 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-6389b7df-b547-4339-8b14-01d884f97529 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:46.842150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:46.844824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:46.846977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:46.846977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:46.846977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:46.846977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:46.846977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:46.846977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:46.846977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:46.846977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:46.846977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:40:46.846977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:46.846977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 479/978] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:40:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:46.849881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0063caa-4085-4e47-8d4d-cfd9ecf241b7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:46.863751 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:46.964547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0063caa-4085-4e47-8d4d-cfd9ecf241b7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:47.002748 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99966c0-3fed-48c0-9365-f5d9897af3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:47.003461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99966c0-3fed-48c0-9365-f5d9897af3a9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:47.003748 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99966c0-3fed-48c0-9365-f5d9897af3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:47.004027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99966c0-3fed-48c0-9365-f5d9897af3a9 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:47.004835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99966c0-3fed-48c0-9365-f5d9897af3a9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:47.005140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99966c0-3fed-48c0-9365-f5d9897af3a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:47.006788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99966c0-3fed-48c0-9365-f5d9897af3a9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:47.047611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 500/979] 10.222.0.121 () {56 vars in 1106 bytes} [Thu May 26 17:40:46 2022] GET /identity/v3/users?name=cinder => generated 356 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:47.158760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c9e33-bf43-4a0d-baa9-6d3240a1fa35 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:47.159132 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c9e33-bf43-4a0d-baa9-6d3240a1fa35 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:47.159349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c9e33-bf43-4a0d-baa9-6d3240a1fa35 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:47.161202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 480/980] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:47 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:47.169189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106a6c5-883a-4c2c-a5d5-190a7efa157f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:47.169189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106a6c5-883a-4c2c-a5d5-190a7efa157f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:47.169189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106a6c5-883a-4c2c-a5d5-190a7efa157f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:47.187783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4106a6c5-883a-4c2c-a5d5-190a7efa157f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:47.189743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4106a6c5-883a-4c2c-a5d5-190a7efa157f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:47.261574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:47.275981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 501/981] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:47 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:47.302550 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c9e33-bf43-4a0d-baa9-6d3240a1fa35 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:47.413523 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c9e33-bf43-4a0d-baa9-6d3240a1fa35 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:47.449040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29ed48-5927-4702-a8b0-bc396af14a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:47.449704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29ed48-5927-4702-a8b0-bc396af14a0e None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:47.449983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29ed48-5927-4702-a8b0-bc396af14a0e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:47.450252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29ed48-5927-4702-a8b0-bc396af14a0e None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:47.451016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29ed48-5927-4702-a8b0-bc396af14a0e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:47.453234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29ed48-5927-4702-a8b0-bc396af14a0e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:47.529598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 481/982] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:40:47 2022] POST /identity/v3/services => generated 243 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:40:47.972401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb072a-df16-4f2a-bef6-101f2f30e924 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:47.972930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb072a-df16-4f2a-bef6-101f2f30e924 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:47.972930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb072a-df16-4f2a-bef6-101f2f30e924 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:47.974450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 502/983] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:47.979641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed0c02-6ba2-4f29-b869-1e7252029a55 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:47.979931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed0c02-6ba2-4f29-b869-1e7252029a55 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:47.980207 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed0c02-6ba2-4f29-b869-1e7252029a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:47.997356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-7aed0c02-6ba2-4f29-b869-1e7252029a55 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:47.998526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7aed0c02-6ba2-4f29-b869-1e7252029a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:48.006203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832d51-896c-40aa-997f-039ce14bce16 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:48.006494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832d51-896c-40aa-997f-039ce14bce16 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:48.006766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832d51-896c-40aa-997f-039ce14bce16 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:48.008304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 503/984] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:48.013010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517264d-298c-4843-b458-c61491b6ea2a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:48.013295 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517264d-298c-4843-b458-c61491b6ea2a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:48.013572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517264d-298c-4843-b458-c61491b6ea2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:48.018777 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:48.031588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e517264d-298c-4843-b458-c61491b6ea2a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:48.032788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e517264d-298c-4843-b458-c61491b6ea2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:48.078440 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 482/985] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:47 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:48.101464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed0c02-6ba2-4f29-b869-1e7252029a55 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:48.113128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 504/986] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:48 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:48.135843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517264d-298c-4843-b458-c61491b6ea2a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:48.208315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed0c02-6ba2-4f29-b869-1e7252029a55 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:48.245600 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469274d3-8e0b-4969-9927-495d9aad44f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:48.246336 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469274d3-8e0b-4969-9927-495d9aad44f2 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:48.246619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469274d3-8e0b-4969-9927-495d9aad44f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:48.246927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469274d3-8e0b-4969-9927-495d9aad44f2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:48.247586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469274d3-8e0b-4969-9927-495d9aad44f2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:48.249019 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469274d3-8e0b-4969-9927-495d9aad44f2 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:48.256548 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517264d-298c-4843-b458-c61491b6ea2a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-469274d3-8e0b-4969-9927-495d9aad44f2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:48.261561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:48.262542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:48.263424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 483/987] 10.222.0.121 () {56 vars in 1131 bytes} [Thu May 26 17:40:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:48.269473 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469274d3-8e0b-4969-9927-495d9aad44f2 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:48.285222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:48.287882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:48.301113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610177c4-2c5f-49aa-bdc5-d8a3d7a46166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:48.301912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610177c4-2c5f-49aa-bdc5-d8a3d7a46166 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:48.302194 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610177c4-2c5f-49aa-bdc5-d8a3d7a46166 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:48.302473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610177c4-2c5f-49aa-bdc5-d8a3d7a46166 None admin] PATH_INFO: `/v3/services/image` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:48.303147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610177c4-2c5f-49aa-bdc5-d8a3d7a46166 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:48.304533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610177c4-2c5f-49aa-bdc5-d8a3d7a46166 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-610177c4-2c5f-49aa-bdc5-d8a3d7a46166 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:48.317440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:48.318688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:48.326006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 505/988] 10.222.0.121 () {56 vars in 1110 bytes} [Thu May 26 17:40:48 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:48.327027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610177c4-2c5f-49aa-bdc5-d8a3d7a46166 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:48.380866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469274d3-8e0b-4969-9927-495d9aad44f2 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:48.422499 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bea2df-c4d0-4588-b424-45a882ad73de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:48.423990 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bea2df-c4d0-4588-b424-45a882ad73de None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:48.424451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bea2df-c4d0-4588-b424-45a882ad73de None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:48.424974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bea2df-c4d0-4588-b424-45a882ad73de None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:48.425940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bea2df-c4d0-4588-b424-45a882ad73de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:48.426405 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bea2df-c4d0-4588-b424-45a882ad73de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:48.428081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bea2df-c4d0-4588-b424-45a882ad73de None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:48.430986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610177c4-2c5f-49aa-bdc5-d8a3d7a46166 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:48.433615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 484/989] 10.222.0.121 () {56 vars in 1127 bytes} [Thu May 26 17:40:48 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:48.441453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bea2df-c4d0-4588-b424-45a882ad73de None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:48.474293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52c47f-e404-416f-b3ce-06da40ef71ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:48.475070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52c47f-e404-416f-b3ce-06da40ef71ba None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:48.475298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52c47f-e404-416f-b3ce-06da40ef71ba None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:48.475567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52c47f-e404-416f-b3ce-06da40ef71ba None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:48.476270 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52c47f-e404-416f-b3ce-06da40ef71ba None admin] RBAC: Adding query filter params (name=image) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:48.476581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52c47f-e404-416f-b3ce-06da40ef71ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:48.478402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52c47f-e404-416f-b3ce-06da40ef71ba None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:48.484493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 506/990] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:48 2022] GET /identity/v3/services?name=image => generated 123 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:48.489890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52c47f-e404-416f-b3ce-06da40ef71ba None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:48.531718 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:48.542029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:48.560523 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bea2df-c4d0-4588-b424-45a882ad73de None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:48.598973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52c47f-e404-416f-b3ce-06da40ef71ba None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:48.602494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2952306-f0c8-4634-8dd4-9620956b3658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:48.603434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2952306-f0c8-4634-8dd4-9620956b3658 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:48.603936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2952306-f0c8-4634-8dd4-9620956b3658 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:48.604435 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2952306-f0c8-4634-8dd4-9620956b3658 None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:48.605337 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2952306-f0c8-4634-8dd4-9620956b3658 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:48.605880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2952306-f0c8-4634-8dd4-9620956b3658 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:48.607445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2952306-f0c8-4634-8dd4-9620956b3658 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:48.615236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 485/991] 10.222.0.121 () {56 vars in 1127 bytes} [Thu May 26 17:40:48 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:48.620744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2952306-f0c8-4634-8dd4-9620956b3658 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:48.634350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290094be-84c0-4006-9ee9-ded5afbb6371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:48.635059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290094be-84c0-4006-9ee9-ded5afbb6371 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:48.635340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290094be-84c0-4006-9ee9-ded5afbb6371 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:48.635617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290094be-84c0-4006-9ee9-ded5afbb6371 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:48.636327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290094be-84c0-4006-9ee9-ded5afbb6371 None admin] RBAC: Adding query filter params (type=image) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:48.636639 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290094be-84c0-4006-9ee9-ded5afbb6371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:48.637996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290094be-84c0-4006-9ee9-ded5afbb6371 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:48.643064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 507/992] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:48 2022] GET /identity/v3/services?type=image => generated 123 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:48.749621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2952306-f0c8-4634-8dd4-9620956b3658 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:48.788888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac085d2a-cc54-4026-a064-5d3d12390734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:48.789487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac085d2a-cc54-4026-a064-5d3d12390734 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:48.789767 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac085d2a-cc54-4026-a064-5d3d12390734 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:48.790036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac085d2a-cc54-4026-a064-5d3d12390734 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:48.791138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac085d2a-cc54-4026-a064-5d3d12390734 None admin] RBAC: Adding query filter params (interface=admin, service_id=e0123a373d7a443f8a9800e7c1c7cdf6, region_id=RegionOne) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:48.791452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac085d2a-cc54-4026-a064-5d3d12390734 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:48.792881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac085d2a-cc54-4026-a064-5d3d12390734 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:48.804890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 486/993] 10.222.0.121 () {56 vars in 1254 bytes} [Thu May 26 17:40:48 2022] GET /identity/v3/endpoints?service_id=e0123a373d7a443f8a9800e7c1c7cdf6&interface=admin®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:48.806438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290094be-84c0-4006-9ee9-ded5afbb6371 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:48.913570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290094be-84c0-4006-9ee9-ded5afbb6371 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:48.927047 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:48.948376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b4422-2268-424c-a12b-855a483ec0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:48.949042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b4422-2268-424c-a12b-855a483ec0cf None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:48.949320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b4422-2268-424c-a12b-855a483ec0cf None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:48.949590 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b4422-2268-424c-a12b-855a483ec0cf None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:48.950263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b4422-2268-424c-a12b-855a483ec0cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:48.951654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b4422-2268-424c-a12b-855a483ec0cf None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:48.959026 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 508/994] 10.222.0.121 () {56 vars in 1092 bytes} [Thu May 26 17:40:48 2022] GET /identity/v3/services => generated 1735 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:40:49.800554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ad759-959d-4c6c-8de0-14265692c674 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:49.801135 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ad759-959d-4c6c-8de0-14265692c674 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:49.801215 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ad759-959d-4c6c-8de0-14265692c674 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:49.802677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 487/995] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:49.807854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff173e2-e2d4-4b31-ba7e-97eeaaf050ed None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:49.808294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff173e2-e2d4-4b31-ba7e-97eeaaf050ed None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:49.808488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff173e2-e2d4-4b31-ba7e-97eeaaf050ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:49.825698 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-0ff173e2-e2d4-4b31-ba7e-97eeaaf050ed None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:49.827645 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0ff173e2-e2d4-4b31-ba7e-97eeaaf050ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:49.896229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 509/996] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:49 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:49.918902 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ad759-959d-4c6c-8de0-14265692c674 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:49.923671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:50.022124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ad759-959d-4c6c-8de0-14265692c674 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:50.057579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f3b0b-e90e-4ddd-b0c4-5641cde2fd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:50.058290 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f3b0b-e90e-4ddd-b0c4-5641cde2fd34 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:50.058567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f3b0b-e90e-4ddd-b0c4-5641cde2fd34 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:50.058857 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f3b0b-e90e-4ddd-b0c4-5641cde2fd34 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:50.059493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f3b0b-e90e-4ddd-b0c4-5641cde2fd34 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:50.060850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f3b0b-e90e-4ddd-b0c4-5641cde2fd34 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-3f8f3b0b-e90e-4ddd-b0c4-5641cde2fd34 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:50.073009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:50.073999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:40:50.075268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:50.076535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 488/997] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:50.081000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff173e2-e2d4-4b31-ba7e-97eeaaf050ed None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:50.192855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff173e2-e2d4-4b31-ba7e-97eeaaf050ed None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:50.200861 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:50.227675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f5633-8e1f-49fa-a947-3a4972544cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:50.228349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f5633-8e1f-49fa-a947-3a4972544cf2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:50.228624 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f5633-8e1f-49fa-a947-3a4972544cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:50.228899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f5633-8e1f-49fa-a947-3a4972544cf2 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:50.229650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f5633-8e1f-49fa-a947-3a4972544cf2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:50.230337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f5633-8e1f-49fa-a947-3a4972544cf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:50.232415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f5633-8e1f-49fa-a947-3a4972544cf2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:50.241670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 510/998] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:50 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:50.243258 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f3b0b-e90e-4ddd-b0c4-5641cde2fd34 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:50.314430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:50.350140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f3b0b-e90e-4ddd-b0c4-5641cde2fd34 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:50.383603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34e896-9858-4c0e-8a7f-4fa5e7fc9968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:50.384265 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34e896-9858-4c0e-8a7f-4fa5e7fc9968 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:50.384548 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34e896-9858-4c0e-8a7f-4fa5e7fc9968 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:50.384820 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34e896-9858-4c0e-8a7f-4fa5e7fc9968 None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:50.385513 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34e896-9858-4c0e-8a7f-4fa5e7fc9968 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:50.385812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34e896-9858-4c0e-8a7f-4fa5e7fc9968 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:50.387182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34e896-9858-4c0e-8a7f-4fa5e7fc9968 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:50.392386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 489/999] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:50 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:50.398507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f5633-8e1f-49fa-a947-3a4972544cf2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:50.509511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f5633-8e1f-49fa-a947-3a4972544cf2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:50.544711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e883d6-6004-43ec-bf88-1f4a66fd6049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:50.545381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e883d6-6004-43ec-bf88-1f4a66fd6049 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:50.545710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e883d6-6004-43ec-bf88-1f4a66fd6049 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:50.546065 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e883d6-6004-43ec-bf88-1f4a66fd6049 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:50.547409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e883d6-6004-43ec-bf88-1f4a66fd6049 None admin] RBAC: Adding query filter params (interface=public, service_id=12e49d37909a42789bdfadca4869069c, region_id=RegionOne) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:50.547521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e883d6-6004-43ec-bf88-1f4a66fd6049 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:50.548878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e883d6-6004-43ec-bf88-1f4a66fd6049 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:50.558544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 511/1000] 10.222.0.121 () {56 vars in 1256 bytes} [Thu May 26 17:40:50 2022] GET /identity/v3/endpoints?service_id=12e49d37909a42789bdfadca4869069c&interface=public®ion_id=RegionOne => generated 195 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:50.560078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34e896-9858-4c0e-8a7f-4fa5e7fc9968 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:50.667373 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34e896-9858-4c0e-8a7f-4fa5e7fc9968 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:50.696397 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:50.702976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b6f95-1fd1-4045-aa8d-c22f6d5b8f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:50.713813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b6f95-1fd1-4045-aa8d-c22f6d5b8f9a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:50.713813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b6f95-1fd1-4045-aa8d-c22f6d5b8f9a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:50.713813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b6f95-1fd1-4045-aa8d-c22f6d5b8f9a None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:50.713813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b6f95-1fd1-4045-aa8d-c22f6d5b8f9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:50.713813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b6f95-1fd1-4045-aa8d-c22f6d5b8f9a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:50.714495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 490/1001] 10.222.0.121 () {56 vars in 1092 bytes} [Thu May 26 17:40:50 2022] GET /identity/v3/services => generated 1735 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:40:50.914079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a92dc-d5cc-4c1c-a9a0-923839e4240c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:50.915068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a92dc-d5cc-4c1c-a9a0-923839e4240c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:50.915808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a92dc-d5cc-4c1c-a9a0-923839e4240c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:50.917331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 512/1002] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:50 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:50.923667 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb3779-1dce-4543-80ea-b8b023371bf7 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:50.924119 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb3779-1dce-4543-80ea-b8b023371bf7 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:50.924198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb3779-1dce-4543-80ea-b8b023371bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:50.941906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b1eb3779-1dce-4543-80ea-b8b023371bf7 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:50.943173 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b1eb3779-1dce-4543-80ea-b8b023371bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:51.015219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 491/1003] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:50 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:51.036770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a92dc-d5cc-4c1c-a9a0-923839e4240c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:51.079367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:51.154865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a92dc-d5cc-4c1c-a9a0-923839e4240c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:51.190371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea779f7-3c73-4605-9c87-c15339e01127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:51.190986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea779f7-3c73-4605-9c87-c15339e01127 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:51.191265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea779f7-3c73-4605-9c87-c15339e01127 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:51.191535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea779f7-3c73-4605-9c87-c15339e01127 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:51.192249 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea779f7-3c73-4605-9c87-c15339e01127 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:51.193605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea779f7-3c73-4605-9c87-c15339e01127 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:51.247224 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74327270-ea98-4a13-92da-e933374d06d6 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:51.247535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74327270-ea98-4a13-92da-e933374d06d6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:51.247839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74327270-ea98-4a13-92da-e933374d06d6 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:51.250100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 492/1004] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:51 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:51.255406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fd7ad-f6b7-4b70-9012-120943e109c3 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:51.255694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fd7ad-f6b7-4b70-9012-120943e109c3 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:51.255969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fd7ad-f6b7-4b70-9012-120943e109c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:51.273027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-379fd7ad-f6b7-4b70-9012-120943e109c3 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:51.274190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-379fd7ad-f6b7-4b70-9012-120943e109c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:51.288864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 513/1005] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:40:51 2022] POST /identity/v3/services => generated 243 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:40:51.364554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 493/1006] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:51 2022] POST /identity/v3/auth/tokens => generated 1658 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:51.388055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea779f7-3c73-4605-9c87-c15339e01127 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:51.497975 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:51.506266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea779f7-3c73-4605-9c87-c15339e01127 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:51.541649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291b3bf-47a9-47b1-ac1c-60de223d5b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:51.542470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291b3bf-47a9-47b1-ac1c-60de223d5b51 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:51.542750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291b3bf-47a9-47b1-ac1c-60de223d5b51 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:51.543051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291b3bf-47a9-47b1-ac1c-60de223d5b51 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:51.543701 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291b3bf-47a9-47b1-ac1c-60de223d5b51 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:51.545104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291b3bf-47a9-47b1-ac1c-60de223d5b51 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-a291b3bf-47a9-47b1-ac1c-60de223d5b51 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:51.558023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:51.559360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:51.560616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 514/1007] 10.222.0.121 () {56 vars in 1131 bytes} [Thu May 26 17:40:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:51.568933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fd7ad-f6b7-4b70-9012-120943e109c3 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:51.622320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:51.671613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fd7ad-f6b7-4b70-9012-120943e109c3 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:51.707574 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789791e-bc1c-4fe9-890d-8b433cf7e167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:51.708246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789791e-bc1c-4fe9-890d-8b433cf7e167 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:51.708527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789791e-bc1c-4fe9-890d-8b433cf7e167 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:51.708800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789791e-bc1c-4fe9-890d-8b433cf7e167 None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:51.709506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789791e-bc1c-4fe9-890d-8b433cf7e167 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:51.709815 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789791e-bc1c-4fe9-890d-8b433cf7e167 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:51.711200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789791e-bc1c-4fe9-890d-8b433cf7e167 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:51.721419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 494/1008] 10.222.0.121 () {56 vars in 1127 bytes} [Thu May 26 17:40:51 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:51.723094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291b3bf-47a9-47b1-ac1c-60de223d5b51 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:51.831880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291b3bf-47a9-47b1-ac1c-60de223d5b51 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:51.873979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0069a17-bf51-4906-b5df-b211a5586e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:51.874718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0069a17-bf51-4906-b5df-b211a5586e84 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:51.875222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0069a17-bf51-4906-b5df-b211a5586e84 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:51.875328 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0069a17-bf51-4906-b5df-b211a5586e84 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:51.876012 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0069a17-bf51-4906-b5df-b211a5586e84 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:51.876328 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0069a17-bf51-4906-b5df-b211a5586e84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:51.877702 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0069a17-bf51-4906-b5df-b211a5586e84 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:51.887446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 515/1009] 10.222.0.121 () {56 vars in 1127 bytes} [Thu May 26 17:40:51 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:51.888712 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789791e-bc1c-4fe9-890d-8b433cf7e167 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:51.995110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789791e-bc1c-4fe9-890d-8b433cf7e167 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:52.011584 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:52.030476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7f006-49ab-4968-8ba2-29909e37250e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:52.031170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7f006-49ab-4968-8ba2-29909e37250e None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:52.031448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7f006-49ab-4968-8ba2-29909e37250e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:52.031717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7f006-49ab-4968-8ba2-29909e37250e None admin] PATH_INFO: `/v3/endpoints` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:52.032456 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7f006-49ab-4968-8ba2-29909e37250e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:52.033819 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7f006-49ab-4968-8ba2-29909e37250e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:52.111314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 495/1010] 10.222.0.121 () {60 vars in 1149 bytes} [Thu May 26 17:40:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:40:52.975270 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa6844-0c4f-40f9-b0fd-16cf75d6c6eb None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:52.975800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa6844-0c4f-40f9-b0fd-16cf75d6c6eb None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:52.975800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa6844-0c4f-40f9-b0fd-16cf75d6c6eb None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:52.977235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 516/1011] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:52.982340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73767cf6-11bf-4974-aa64-2df3a73083d5 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:52.982699 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73767cf6-11bf-4974-aa64-2df3a73083d5 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:52.983621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73767cf6-11bf-4974-aa64-2df3a73083d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:53.001194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-73767cf6-11bf-4974-aa64-2df3a73083d5 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:53.002360 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-73767cf6-11bf-4974-aa64-2df3a73083d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:53.079098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 496/1012] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:52 2022] POST /identity/v3/auth/tokens => generated 1806 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:53.106529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa6844-0c4f-40f9-b0fd-16cf75d6c6eb None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:53.135618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:53.218946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa6844-0c4f-40f9-b0fd-16cf75d6c6eb None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:53.256990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be06f8a-4994-4209-9a72-dbd4ded3a10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:53.257714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be06f8a-4994-4209-9a72-dbd4ded3a10a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:53.258000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be06f8a-4994-4209-9a72-dbd4ded3a10a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:53.258278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be06f8a-4994-4209-9a72-dbd4ded3a10a None admin] PATH_INFO: `/v3/services/compute` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:53.258957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be06f8a-4994-4209-9a72-dbd4ded3a10a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:53.260334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be06f8a-4994-4209-9a72-dbd4ded3a10a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-3be06f8a-4994-4209-9a72-dbd4ded3a10a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:53.272829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:53.273833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:53.274793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 517/1013] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:53.282929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73767cf6-11bf-4974-aa64-2df3a73083d5 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:53.386258 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73767cf6-11bf-4974-aa64-2df3a73083d5 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:53.387854 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:53.427060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd190796-450d-4490-962f-e0558e21443c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:53.427752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190796-450d-4490-962f-e0558e21443c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:53.428030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190796-450d-4490-962f-e0558e21443c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:53.428305 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190796-450d-4490-962f-e0558e21443c None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:53.429023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd190796-450d-4490-962f-e0558e21443c None admin] RBAC: Adding query filter params (name=compute) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:53.429329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd190796-450d-4490-962f-e0558e21443c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:53.430713 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd190796-450d-4490-962f-e0558e21443c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:53.441097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 497/1014] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:53 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:53.442313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be06f8a-4994-4209-9a72-dbd4ded3a10a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:53.532528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:53.549293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be06f8a-4994-4209-9a72-dbd4ded3a10a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:53.585508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58101f-c5ae-447a-a05e-4c498b001870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:53.586171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58101f-c5ae-447a-a05e-4c498b001870 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:53.586448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58101f-c5ae-447a-a05e-4c498b001870 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:53.586715 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58101f-c5ae-447a-a05e-4c498b001870 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:53.587441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58101f-c5ae-447a-a05e-4c498b001870 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:53.587745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58101f-c5ae-447a-a05e-4c498b001870 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:53.589104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58101f-c5ae-447a-a05e-4c498b001870 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:53.594122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 518/1015] 10.222.0.121 () {56 vars in 1117 bytes} [Thu May 26 17:40:53 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:53.598626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769aa7d-9f53-4b1f-a164-6fc76b285fde None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:53.598948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769aa7d-9f53-4b1f-a164-6fc76b285fde None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:53.599226 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769aa7d-9f53-4b1f-a164-6fc76b285fde None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:53.599323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58101f-c5ae-447a-a05e-4c498b001870 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:53.600691 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 498/1016] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:53.605466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62afbf-1d29-41b2-a9d8-04a24233b046 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:53.605749 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62afbf-1d29-41b2-a9d8-04a24233b046 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:53.606021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62afbf-1d29-41b2-a9d8-04a24233b046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:53.623471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3d62afbf-1d29-41b2-a9d8-04a24233b046 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:53.625357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3d62afbf-1d29-41b2-a9d8-04a24233b046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:53.697649 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 499/1017] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:53 2022] POST /identity/v3/auth/tokens => generated 1806 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:53.707556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58101f-c5ae-447a-a05e-4c498b001870 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:53.721763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62afbf-1d29-41b2-a9d8-04a24233b046 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:53.750299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edde694-bd5c-45b2-8b96-de09a144d642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:53.751067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edde694-bd5c-45b2-8b96-de09a144d642 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:53.752079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edde694-bd5c-45b2-8b96-de09a144d642 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:53.752079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edde694-bd5c-45b2-8b96-de09a144d642 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:53.752560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edde694-bd5c-45b2-8b96-de09a144d642 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:53.753943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edde694-bd5c-45b2-8b96-de09a144d642 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:53.822236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:53.829563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62afbf-1d29-41b2-a9d8-04a24233b046 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:53.863326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 519/1018] 10.222.0.121 () {60 vars in 1149 bytes} [Thu May 26 17:40:53 2022] POST /identity/v3/endpoints => generated 336 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:40:53.870538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e8ce9-19b4-454c-a65f-57ef833cea46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:53.871383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e8ce9-19b4-454c-a65f-57ef833cea46 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:53.871660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e8ce9-19b4-454c-a65f-57ef833cea46 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:53.871924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e8ce9-19b4-454c-a65f-57ef833cea46 None admin] PATH_INFO: `/v3/services/image` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:53.872789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e8ce9-19b4-454c-a65f-57ef833cea46 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:53.874739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e8ce9-19b4-454c-a65f-57ef833cea46 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-e02e8ce9-19b4-454c-a65f-57ef833cea46 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:53.888595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:53.890007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:53.890882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 500/1019] 10.222.0.121 () {56 vars in 1110 bytes} [Thu May 26 17:40:53 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:53.898087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edde694-bd5c-45b2-8b96-de09a144d642 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:53.908385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:54.012962 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edde694-bd5c-45b2-8b96-de09a144d642 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:54.052032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2cc8d-65a5-4257-a27e-5b6d2ab0ad8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:54.052712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2cc8d-65a5-4257-a27e-5b6d2ab0ad8a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:54.052997 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2cc8d-65a5-4257-a27e-5b6d2ab0ad8a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:54.053272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2cc8d-65a5-4257-a27e-5b6d2ab0ad8a None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:54.053979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2cc8d-65a5-4257-a27e-5b6d2ab0ad8a None admin] RBAC: Adding query filter params (name=image) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:54.054290 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2cc8d-65a5-4257-a27e-5b6d2ab0ad8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:54.055702 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2cc8d-65a5-4257-a27e-5b6d2ab0ad8a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:54.061040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 520/1020] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:53 2022] GET /identity/v3/services?name=image => generated 123 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:54.066994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e8ce9-19b4-454c-a65f-57ef833cea46 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:54.190134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e8ce9-19b4-454c-a65f-57ef833cea46 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:54.228448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73762b18-63e4-4ab6-b1bc-8eb76221a3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:54.229124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73762b18-63e4-4ab6-b1bc-8eb76221a3bc None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:54.229407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73762b18-63e4-4ab6-b1bc-8eb76221a3bc None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:54.229684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73762b18-63e4-4ab6-b1bc-8eb76221a3bc None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:54.230388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73762b18-63e4-4ab6-b1bc-8eb76221a3bc None admin] RBAC: Adding query filter params (type=image) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:54.230695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73762b18-63e4-4ab6-b1bc-8eb76221a3bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:54.232162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73762b18-63e4-4ab6-b1bc-8eb76221a3bc None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:54.240577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 501/1021] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:54 2022] GET /identity/v3/services?type=image => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:54.246102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2cc8d-65a5-4257-a27e-5b6d2ab0ad8a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:54.323078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:54.355634 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2cc8d-65a5-4257-a27e-5b6d2ab0ad8a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:54.378384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe75f6-e959-4330-9557-ac4508616dc7 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:54.378715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe75f6-e959-4330-9557-ac4508616dc7 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:54.378978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe75f6-e959-4330-9557-ac4508616dc7 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:54.380499 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 502/1022] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:54.385260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e574dc9-9739-4d90-b5fb-28b6dbeb7a9c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:54.385585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e574dc9-9739-4d90-b5fb-28b6dbeb7a9c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:54.385869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e574dc9-9739-4d90-b5fb-28b6dbeb7a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:54.393277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ebe6e-c94d-45cf-961e-f5ceaad66a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:54.393948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ebe6e-c94d-45cf-961e-f5ceaad66a33 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:54.394231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ebe6e-c94d-45cf-961e-f5ceaad66a33 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:54.394504 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ebe6e-c94d-45cf-961e-f5ceaad66a33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:54.395324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ebe6e-c94d-45cf-961e-f5ceaad66a33 None admin] RBAC: Adding query filter params (interface=public, service_id=0a5e2a4c28b2483cbb145fd678fa8127, region_id=RegionOne) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:54.395640 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ebe6e-c94d-45cf-961e-f5ceaad66a33 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:54.397023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ebe6e-c94d-45cf-961e-f5ceaad66a33 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:54.407438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 521/1023] 10.222.0.121 () {56 vars in 1256 bytes} [Thu May 26 17:40:54 2022] GET /identity/v3/endpoints?service_id=0a5e2a4c28b2483cbb145fd678fa8127&interface=public®ion_id=RegionOne => generated 195 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:54.407623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-2e574dc9-9739-4d90-b5fb-28b6dbeb7a9c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:54.407623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-2e574dc9-9739-4d90-b5fb-28b6dbeb7a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:54.409282 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ebe6e-c94d-45cf-961e-f5ceaad66a33 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:54.422536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:54.491144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 503/1024] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:54 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:54.504814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e574dc9-9739-4d90-b5fb-28b6dbeb7a9c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:54.519880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ebe6e-c94d-45cf-961e-f5ceaad66a33 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:54.556502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fcd18-d290-49e4-b618-0964bcfe98c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:54.557176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fcd18-d290-49e4-b618-0964bcfe98c8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:54.557464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fcd18-d290-49e4-b618-0964bcfe98c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:54.557730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fcd18-d290-49e4-b618-0964bcfe98c8 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:54.558401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fcd18-d290-49e4-b618-0964bcfe98c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:54.559798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fcd18-d290-49e4-b618-0964bcfe98c8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:54.561078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:54.574854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 522/1025] 10.222.0.121 () {56 vars in 1092 bytes} [Thu May 26 17:40:54 2022] GET /identity/v3/services => generated 1966 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:40:54.634220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e574dc9-9739-4d90-b5fb-28b6dbeb7a9c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:54.672842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315be2a0-8d95-4ac5-af97-9de3775f65f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:54.673659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315be2a0-8d95-4ac5-af97-9de3775f65f5 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:54.673941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315be2a0-8d95-4ac5-af97-9de3775f65f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:54.674211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315be2a0-8d95-4ac5-af97-9de3775f65f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:54.675530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315be2a0-8d95-4ac5-af97-9de3775f65f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:54.676992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315be2a0-8d95-4ac5-af97-9de3775f65f5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:54.678653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 504/1026] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:54 2022] GET /identity/v3/domains/default => generated 200 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:54.684641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fcd18-d290-49e4-b618-0964bcfe98c8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:54.795482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fcd18-d290-49e4-b618-0964bcfe98c8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:54.832450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665b6d9-ff57-493a-b167-74f957496bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:54.833126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b6d9-ff57-493a-b167-74f957496bb2 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:54.833402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b6d9-ff57-493a-b167-74f957496bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:54.833673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b6d9-ff57-493a-b167-74f957496bb2 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:54.834382 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665b6d9-ff57-493a-b167-74f957496bb2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:54.835835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665b6d9-ff57-493a-b167-74f957496bb2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:54.925640 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 523/1027] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:40:54 2022] POST /identity/v3/projects => generated 312 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:40:56.265288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8436bd-602f-439c-b53a-0648b5477be5 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:56.265917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8436bd-602f-439c-b53a-0648b5477be5 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:56.265917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8436bd-602f-439c-b53a-0648b5477be5 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:56.267419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 505/1028] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:56.272504 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068584d-ea25-42d4-97f7-4e747ca6e2e5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:56.272905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068584d-ea25-42d4-97f7-4e747ca6e2e5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:56.273068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068584d-ea25-42d4-97f7-4e747ca6e2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:56.290505 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d068584d-ea25-42d4-97f7-4e747ca6e2e5 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:56.291719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d068584d-ea25-42d4-97f7-4e747ca6e2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:56.339523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:56.362588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 524/1029] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:56 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:56.387046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8436bd-602f-439c-b53a-0648b5477be5 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:56.408091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:56.497808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8436bd-602f-439c-b53a-0648b5477be5 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:56.537817 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1dd18-84a9-4cea-8503-06769ab31033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:56.538631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1dd18-84a9-4cea-8503-06769ab31033 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:56.538937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1dd18-84a9-4cea-8503-06769ab31033 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:56.539209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1dd18-84a9-4cea-8503-06769ab31033 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:56.550348 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1dd18-84a9-4cea-8503-06769ab31033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-1ca1dd18-84a9-4cea-8503-06769ab31033 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:40:56.558434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:56.559416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:56.565871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 506/1030] 10.222.0.121 () {56 vars in 1125 bytes} [Thu May 26 17:40:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:56.566584 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068584d-ea25-42d4-97f7-4e747ca6e2e5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:56.670129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068584d-ea25-42d4-97f7-4e747ca6e2e5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:56.712246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca6c51-e931-4a56-942a-d214396f1844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:56.712927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca6c51-e931-4a56-942a-d214396f1844 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:56.713210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca6c51-e931-4a56-942a-d214396f1844 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:56.713483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca6c51-e931-4a56-942a-d214396f1844 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:56.714228 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca6c51-e931-4a56-942a-d214396f1844 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:56.714533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca6c51-e931-4a56-942a-d214396f1844 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:56.715939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca6c51-e931-4a56-942a-d214396f1844 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:56.730868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 525/1031] 10.222.0.121 () {56 vars in 1120 bytes} [Thu May 26 17:40:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:56.732672 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1dd18-84a9-4cea-8503-06769ab31033 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:56.821617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:56.844156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1dd18-84a9-4cea-8503-06769ab31033 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:56.869870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d38ad7-dd20-418e-9ae9-d227411702d1 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:56.870155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d38ad7-dd20-418e-9ae9-d227411702d1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:56.870432 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d38ad7-dd20-418e-9ae9-d227411702d1 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:56.871842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 526/1032] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:40:56.876973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5adaeb-2ca6-45a5-a6fd-b496c708cdd6 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:56.877416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5adaeb-2ca6-45a5-a6fd-b496c708cdd6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:56.877878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5adaeb-2ca6-45a5-a6fd-b496c708cdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:56.883598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681d791-f5b6-4762-ab31-25feb9c79386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:56.884899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681d791-f5b6-4762-ab31-25feb9c79386 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:56.884899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681d791-f5b6-4762-ab31-25feb9c79386 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:56.884899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681d791-f5b6-4762-ab31-25feb9c79386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:56.897804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2a5adaeb-2ca6-45a5-a6fd-b496c708cdd6 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:56.899034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2a5adaeb-2ca6-45a5-a6fd-b496c708cdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:56.899172 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681d791-f5b6-4762-ab31-25feb9c79386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:56.900625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681d791-f5b6-4762-ab31-25feb9c79386 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-8681d791-f5b6-4762-ab31-25feb9c79386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:56.919087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:56.920633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:56.922331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:56.922331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:56.922331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:56.922331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:56.925714 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 507/1033] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:56.926613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681d791-f5b6-4762-ab31-25feb9c79386 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:56.940721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:56.983331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 527/1034] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:56 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:40:56.999123 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5adaeb-2ca6-45a5-a6fd-b496c708cdd6 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:57.042303 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681d791-f5b6-4762-ab31-25feb9c79386 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:57.072808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:57.080708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73abb05-16c1-48b0-9c12-03174b793cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:57.081384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73abb05-16c1-48b0-9c12-03174b793cc5 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:57.084409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73abb05-16c1-48b0-9c12-03174b793cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:57.084409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73abb05-16c1-48b0-9c12-03174b793cc5 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:57.084409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73abb05-16c1-48b0-9c12-03174b793cc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:57.084409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73abb05-16c1-48b0-9c12-03174b793cc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:57.084409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73abb05-16c1-48b0-9c12-03174b793cc5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:57.098693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 508/1035] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:57.106455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73abb05-16c1-48b0-9c12-03174b793cc5 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:57.131257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5adaeb-2ca6-45a5-a6fd-b496c708cdd6 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:57.169167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a6007-76b2-4096-ada2-b3a65b130867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:57.169970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a6007-76b2-4096-ada2-b3a65b130867 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:57.170281 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a6007-76b2-4096-ada2-b3a65b130867 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:57.170571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a6007-76b2-4096-ada2-b3a65b130867 None admin] PATH_INFO: `/v3/services/image` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:57.171334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a6007-76b2-4096-ada2-b3a65b130867 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:57.172629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a6007-76b2-4096-ada2-b3a65b130867 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-d30a6007-76b2-4096-ada2-b3a65b130867 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:57.184525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:57.185556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:57.191835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 528/1036] 10.222.0.121 () {56 vars in 1110 bytes} [Thu May 26 17:40:56 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:57.193486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a6007-76b2-4096-ada2-b3a65b130867 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:57.217896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73abb05-16c1-48b0-9c12-03174b793cc5 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:57.229958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:57.254276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e841b65-19bd-4db1-a05e-300c6360e35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:57.255096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e841b65-19bd-4db1-a05e-300c6360e35c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:57.255375 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e841b65-19bd-4db1-a05e-300c6360e35c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:57.255759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e841b65-19bd-4db1-a05e-300c6360e35c None admin] PATH_INFO: `/v3/users/nova` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:57.262905 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e841b65-19bd-4db1-a05e-300c6360e35c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:57.264490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e841b65-19bd-4db1-a05e-300c6360e35c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-8e841b65-19bd-4db1-a05e-300c6360e35c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:40:57.279650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:57.282266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 509/1037] 10.222.0.121 () {56 vars in 1098 bytes} [Thu May 26 17:40:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60628648-00a1-403c-b652-6b1c6621753c None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:57.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60628648-00a1-403c-b652-6b1c6621753c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:57.284422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60628648-00a1-403c-b652-6b1c6621753c None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:57.284422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 510/1038] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:40:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:40:57.286451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60628648-00a1-403c-b652-6b1c6621753c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:57.300112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a6007-76b2-4096-ada2-b3a65b130867 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:57.334303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c2878-d4de-4e75-a6d6-3428babd3aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:57.335001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c2878-d4de-4e75-a6d6-3428babd3aa3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:57.335284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c2878-d4de-4e75-a6d6-3428babd3aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:57.335570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c2878-d4de-4e75-a6d6-3428babd3aa3 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:57.336254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c2878-d4de-4e75-a6d6-3428babd3aa3 None admin] RBAC: Adding query filter params (name=image) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:57.336556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c2878-d4de-4e75-a6d6-3428babd3aa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:57.337896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:57.338013 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c2878-d4de-4e75-a6d6-3428babd3aa3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:57.342771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 529/1039] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:57 2022] GET /identity/v3/services?name=image => generated 123 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:57.344216 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe23c66-d650-427b-a001-c42c930cfa0d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:57.344511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe23c66-d650-427b-a001-c42c930cfa0d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:57.344786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe23c66-d650-427b-a001-c42c930cfa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:57.361325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-efe23c66-d650-427b-a001-c42c930cfa0d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:40:57.362514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-efe23c66-d650-427b-a001-c42c930cfa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:40:57.389437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60628648-00a1-403c-b652-6b1c6621753c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:57.423181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85daee-a1f2-4e1c-b050-8401dfbdb389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:57.423773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85daee-a1f2-4e1c-b050-8401dfbdb389 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:57.424053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85daee-a1f2-4e1c-b050-8401dfbdb389 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:57.424324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85daee-a1f2-4e1c-b050-8401dfbdb389 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:57.425142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85daee-a1f2-4e1c-b050-8401dfbdb389 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:57.425444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85daee-a1f2-4e1c-b050-8401dfbdb389 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:57.426874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85daee-a1f2-4e1c-b050-8401dfbdb389 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:57.429268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 530/1040] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:40:57 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:40:57.430101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe23c66-d650-427b-a001-c42c930cfa0d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:57.446345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 511/1041] 10.222.0.121 () {56 vars in 1138 bytes} [Thu May 26 17:40:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:40:57.453304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85daee-a1f2-4e1c-b050-8401dfbdb389 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:57.470959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:57.543300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe23c66-d650-427b-a001-c42c930cfa0d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:57.559917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85daee-a1f2-4e1c-b050-8401dfbdb389 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:57.569958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:57.580206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293debb6-6a0a-42cd-9dea-f6b815f6650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:57.580870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293debb6-6a0a-42cd-9dea-f6b815f6650a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:57.581153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293debb6-6a0a-42cd-9dea-f6b815f6650a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:57.581428 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293debb6-6a0a-42cd-9dea-f6b815f6650a None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:57.582146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293debb6-6a0a-42cd-9dea-f6b815f6650a None admin] RBAC: Adding query filter params (type=image) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:57.582452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293debb6-6a0a-42cd-9dea-f6b815f6650a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:57.583858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293debb6-6a0a-42cd-9dea-f6b815f6650a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:57.589598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 531/1042] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:57 2022] GET /identity/v3/services?type=image => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:57.590379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293debb6-6a0a-42cd-9dea-f6b815f6650a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:57.594764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00919d-e64a-44c5-a666-f4acb8e48461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:57.602960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00919d-e64a-44c5-a666-f4acb8e48461 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:57.602960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00919d-e64a-44c5-a666-f4acb8e48461 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:57.602960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00919d-e64a-44c5-a666-f4acb8e48461 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:57.679873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00919d-e64a-44c5-a666-f4acb8e48461 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:57.682262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00919d-e64a-44c5-a666-f4acb8e48461 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:57.704285 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:57.710840 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293debb6-6a0a-42cd-9dea-f6b815f6650a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:57.748718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a772c-625d-43d8-bb6f-0cb8484ac936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:57.749745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a772c-625d-43d8-bb6f-0cb8484ac936 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:57.750021 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a772c-625d-43d8-bb6f-0cb8484ac936 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:57.750293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a772c-625d-43d8-bb6f-0cb8484ac936 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:57.751465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a772c-625d-43d8-bb6f-0cb8484ac936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:57.752898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a772c-625d-43d8-bb6f-0cb8484ac936 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:57.754545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 532/1043] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:57 2022] GET /identity/v3/domains/default => generated 200 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:57.755315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a772c-625d-43d8-bb6f-0cb8484ac936 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:57.793129 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 512/1044] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:40:57 2022] GET /identity/v3/auth/tokens => generated 1963 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 17:40:57.793129 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00919d-e64a-44c5-a666-f4acb8e48461 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:57.806722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:57.869145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a772c-625d-43d8-bb6f-0cb8484ac936 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:57.899100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00919d-e64a-44c5-a666-f4acb8e48461 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:57.902726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa197863-7785-4e46-ad12-f8aa98e6cc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:57.903425 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa197863-7785-4e46-ad12-f8aa98e6cc90 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:57.903706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa197863-7785-4e46-ad12-f8aa98e6cc90 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:57.903980 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa197863-7785-4e46-ad12-f8aa98e6cc90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:57.905256 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa197863-7785-4e46-ad12-f8aa98e6cc90 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:57.906623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa197863-7785-4e46-ad12-f8aa98e6cc90 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:57.934428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ec256-b181-44fa-8fa7-e9832ae25bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:57.935088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ec256-b181-44fa-8fa7-e9832ae25bd5 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:57.935438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ec256-b181-44fa-8fa7-e9832ae25bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:57.935711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ec256-b181-44fa-8fa7-e9832ae25bd5 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:57.936628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ec256-b181-44fa-8fa7-e9832ae25bd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:57.938087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ec256-b181-44fa-8fa7-e9832ae25bd5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:58.000001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 533/1045] 10.222.0.121 () {60 vars in 1149 bytes} [Thu May 26 17:40:57 2022] POST /identity/v3/endpoints => generated 329 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:40:58.000905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa197863-7785-4e46-ad12-f8aa98e6cc90 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:58.025345 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:58.026406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 513/1046] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:40:57 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:40:58.144812 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa197863-7785-4e46-ad12-f8aa98e6cc90 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:58.189780 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70fa12-2648-48cb-a821-6b34b6157e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:58.190446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70fa12-2648-48cb-a821-6b34b6157e63 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:58.190723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70fa12-2648-48cb-a821-6b34b6157e63 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:58.191020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70fa12-2648-48cb-a821-6b34b6157e63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:58.202742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70fa12-2648-48cb-a821-6b34b6157e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:58.204515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70fa12-2648-48cb-a821-6b34b6157e63 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-1b70fa12-2648-48cb-a821-6b34b6157e63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:40:58.223985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:58.225175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:58.226517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:40:58.226517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:40:58.226517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:40:58.226517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:58.226517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 534/1047] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:40:58.231926 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ec256-b181-44fa-8fa7-e9832ae25bd5 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:58.305703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:58.351214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ec256-b181-44fa-8fa7-e9832ae25bd5 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:58.390840 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905fce8-9d8e-497e-b9f4-0494ad3e8bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:58.391563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905fce8-9d8e-497e-b9f4-0494ad3e8bf7 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:58.391861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905fce8-9d8e-497e-b9f4-0494ad3e8bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:58.392609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905fce8-9d8e-497e-b9f4-0494ad3e8bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:58.393457 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905fce8-9d8e-497e-b9f4-0494ad3e8bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:58.393771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905fce8-9d8e-497e-b9f4-0494ad3e8bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:58.395201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905fce8-9d8e-497e-b9f4-0494ad3e8bf7 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:58.409143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 514/1048] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:40:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:58.417976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70fa12-2648-48cb-a821-6b34b6157e63 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:58.495495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:58.537828 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70fa12-2648-48cb-a821-6b34b6157e63 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:58.575684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5343e1-f648-4228-acad-749b10fa833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:58.576482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5343e1-f648-4228-acad-749b10fa833d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:58.576764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5343e1-f648-4228-acad-749b10fa833d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:58.577041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5343e1-f648-4228-acad-749b10fa833d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:58.584440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5343e1-f648-4228-acad-749b10fa833d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:58.585887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5343e1-f648-4228-acad-749b10fa833d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-3e5343e1-f648-4228-acad-749b10fa833d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:40:58.600622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:40:58.601585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:40:58.602501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:40:58.602501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:40:58.602501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:40:58.602501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:40:58.602501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:40:58.602501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:40:58.602501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:40:58.602501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:40:58.606500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 535/1049] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:40:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:40:58.607688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905fce8-9d8e-497e-b9f4-0494ad3e8bf7 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:58.716108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905fce8-9d8e-497e-b9f4-0494ad3e8bf7 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:58.741459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:58.754586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5639e4f-589c-4ff6-b20d-226bca94399d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:58.754586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5639e4f-589c-4ff6-b20d-226bca94399d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:58.754586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5639e4f-589c-4ff6-b20d-226bca94399d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:58.755120 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5639e4f-589c-4ff6-b20d-226bca94399d None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:58.755450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5639e4f-589c-4ff6-b20d-226bca94399d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:58.755759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5639e4f-589c-4ff6-b20d-226bca94399d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:58.757190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5639e4f-589c-4ff6-b20d-226bca94399d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:58.771168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 515/1050] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:40:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:40:58.776885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5343e1-f648-4228-acad-749b10fa833d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:40:58.885432 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5343e1-f648-4228-acad-749b10fa833d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:40:58.911421 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:40:58.922961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c930c-74bf-4bd2-9cd7-1d970ce9673f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:40:58.926864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c930c-74bf-4bd2-9cd7-1d970ce9673f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:40:58.926864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c930c-74bf-4bd2-9cd7-1d970ce9673f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:40:58.926864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c930c-74bf-4bd2-9cd7-1d970ce9673f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:40:58.926864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c930c-74bf-4bd2-9cd7-1d970ce9673f None admin] RBAC: Adding query filter params (role.id=1808269227c74f569faa53261e45ca40, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=c0889a69a018452d8de1924072739d17) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:40:58.926864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c930c-74bf-4bd2-9cd7-1d970ce9673f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:40:58.926864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c930c-74bf-4bd2-9cd7-1d970ce9673f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:40:58.937505 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 536/1051] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:40:58 2022] GET /identity/v3/role_assignments?user.id=c0889a69a018452d8de1924072739d17&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=1808269227c74f569faa53261e45ca40 => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:00.297407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1afa3-e62a-4704-99d9-a365481d7a6a None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:00.297931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1afa3-e62a-4704-99d9-a365481d7a6a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:00.298200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1afa3-e62a-4704-99d9-a365481d7a6a None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:00.299649 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 516/1052] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:00 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:00.305032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475fcd7-7d6b-4892-b5e3-c11c540e116d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:00.305455 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475fcd7-7d6b-4892-b5e3-c11c540e116d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:00.305630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475fcd7-7d6b-4892-b5e3-c11c540e116d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:00.322518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f475fcd7-7d6b-4892-b5e3-c11c540e116d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:00.323723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f475fcd7-7d6b-4892-b5e3-c11c540e116d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:00.392223 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b0ae6-1724-46d1-8d65-0c40f155fd66 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:00.392502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b0ae6-1724-46d1-8d65-0c40f155fd66 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:00.392779 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b0ae6-1724-46d1-8d65-0c40f155fd66 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:00.394219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 517/1053] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:00 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:00.399156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 537/1054] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:00 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:00.399379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e93b7-0291-4827-a08a-8ae849b7b461 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:00.399628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e93b7-0291-4827-a08a-8ae849b7b461 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:00.399907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e93b7-0291-4827-a08a-8ae849b7b461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:00.417935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-4c3e93b7-0291-4827-a08a-8ae849b7b461 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:00.419336 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4c3e93b7-0291-4827-a08a-8ae849b7b461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:00.423709 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475fcd7-7d6b-4892-b5e3-c11c540e116d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:00.502468 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 518/1055] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:00 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:41:00.526212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e93b7-0291-4827-a08a-8ae849b7b461 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:00.534799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475fcd7-7d6b-4892-b5e3-c11c540e116d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:00.557082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:00.573904 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88639e79-3093-456b-b1ee-109bca106306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:00.580229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88639e79-3093-456b-b1ee-109bca106306 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:00.580229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88639e79-3093-456b-b1ee-109bca106306 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:00.580229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88639e79-3093-456b-b1ee-109bca106306 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:00.634770 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e93b7-0291-4827-a08a-8ae849b7b461 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:00.646649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88639e79-3093-456b-b1ee-109bca106306 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:00.648117 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88639e79-3093-456b-b1ee-109bca106306 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:00.651793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:00.673478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429ffb8-f927-4ab7-9134-9a93d099618f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:00.674291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429ffb8-f927-4ab7-9134-9a93d099618f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:00.674573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429ffb8-f927-4ab7-9134-9a93d099618f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:00.674867 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429ffb8-f927-4ab7-9134-9a93d099618f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:00.687445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429ffb8-f927-4ab7-9134-9a93d099618f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-8429ffb8-f927-4ab7-9134-9a93d099618f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:41:00.695238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:00.696254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:41:00.697287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 519/1056] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:41:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:41:00.702800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429ffb8-f927-4ab7-9134-9a93d099618f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:00.760634 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 538/1057] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:41:00 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 17:41:00.766416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88639e79-3093-456b-b1ee-109bca106306 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:00.789924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:00.812563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429ffb8-f927-4ab7-9134-9a93d099618f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:00.848629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5a330-df82-4d5f-bbc2-05554441711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:00.849426 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5a330-df82-4d5f-bbc2-05554441711d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:00.849711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5a330-df82-4d5f-bbc2-05554441711d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:00.849987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5a330-df82-4d5f-bbc2-05554441711d None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:00.850730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5a330-df82-4d5f-bbc2-05554441711d None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:00.851078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5a330-df82-4d5f-bbc2-05554441711d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:00.852453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5a330-df82-4d5f-bbc2-05554441711d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:00.866108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 520/1058] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:41:00 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:00.873099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5a330-df82-4d5f-bbc2-05554441711d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:00.877462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88639e79-3093-456b-b1ee-109bca106306 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:00.912961 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:00.915579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93476e96-2585-4250-804c-ce401be1df63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:00.916378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93476e96-2585-4250-804c-ce401be1df63 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:00.916656 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93476e96-2585-4250-804c-ce401be1df63 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:00.916926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93476e96-2585-4250-804c-ce401be1df63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:00.929281 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93476e96-2585-4250-804c-ce401be1df63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:00.930747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93476e96-2585-4250-804c-ce401be1df63 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-93476e96-2585-4250-804c-ce401be1df63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:41:00.949897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:00.951107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:00.952937 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:41:00.952937 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:41:00.952937 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:41:00.952937 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:00.956642 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 539/1059] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:41:00.957333 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93476e96-2585-4250-804c-ce401be1df63 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:00.989530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5a330-df82-4d5f-bbc2-05554441711d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:01.030646 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2fc3b-4195-499a-8329-76ec1519541b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:01.031563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2fc3b-4195-499a-8329-76ec1519541b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:01.031830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2fc3b-4195-499a-8329-76ec1519541b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:01.032115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2fc3b-4195-499a-8329-76ec1519541b None admin] PATH_INFO: `/v3/users/139081ac5e8546cb959ab468ec59722c` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:01.045678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:01.048267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2fc3b-4195-499a-8329-76ec1519541b None admin] RBAC: Authorizing `identity:get_user(user_id=139081ac5e8546cb959ab468ec59722c)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:01.049719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2fc3b-4195-499a-8329-76ec1519541b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:01.051640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 521/1060] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:00 2022] GET /identity/v3/users/139081ac5e8546cb959ab468ec59722c => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:01.057529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2fc3b-4195-499a-8329-76ec1519541b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:01.083819 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93476e96-2585-4250-804c-ce401be1df63 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:01.127473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c61c25-e4a5-4724-af69-c916290d861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:01.128152 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c61c25-e4a5-4724-af69-c916290d861a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:01.128435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c61c25-e4a5-4724-af69-c916290d861a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:01.128710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c61c25-e4a5-4724-af69-c916290d861a None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:01.129430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c61c25-e4a5-4724-af69-c916290d861a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:01.129958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c61c25-e4a5-4724-af69-c916290d861a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:01.131112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c61c25-e4a5-4724-af69-c916290d861a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:01.144364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 540/1061] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:41:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:01.165508 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:01.173673 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2fc3b-4195-499a-8329-76ec1519541b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:01.178531 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a098-b7ee-49ee-9f2e-e8a3ec21b2af None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:01.187894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a098-b7ee-49ee-9f2e-e8a3ec21b2af None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:01.187894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a098-b7ee-49ee-9f2e-e8a3ec21b2af None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:01.187894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 541/1062] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:41:01.187894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0adccb-fbcc-40a9-b43f-1f65da36d777 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:01.187894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0adccb-fbcc-40a9-b43f-1f65da36d777 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:01.187894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0adccb-fbcc-40a9-b43f-1f65da36d777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:01.205046 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5b0adccb-fbcc-40a9-b43f-1f65da36d777 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:01.206272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5b0adccb-fbcc-40a9-b43f-1f65da36d777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:01.214568 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653eccd6-2b2c-4e67-915c-3640a6382415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:01.215447 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653eccd6-2b2c-4e67-915c-3640a6382415 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:01.215801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653eccd6-2b2c-4e67-915c-3640a6382415 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:01.216074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653eccd6-2b2c-4e67-915c-3640a6382415 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:01.289794 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 542/1063] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:01 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:41:01.299163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653eccd6-2b2c-4e67-915c-3640a6382415 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:01.301290 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653eccd6-2b2c-4e67-915c-3640a6382415 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:01.314352 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0adccb-fbcc-40a9-b43f-1f65da36d777 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:01.396716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:01.420153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:01.430858 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 522/1064] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:41:01 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 17:41:01.439160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653eccd6-2b2c-4e67-915c-3640a6382415 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:01.441340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0adccb-fbcc-40a9-b43f-1f65da36d777 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:01.482504 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5153f99-170c-4d49-b9fb-e35d2a82e59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:01.483392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5153f99-170c-4d49-b9fb-e35d2a82e59e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:01.484070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5153f99-170c-4d49-b9fb-e35d2a82e59e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:01.484349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5153f99-170c-4d49-b9fb-e35d2a82e59e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:01.495030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5153f99-170c-4d49-b9fb-e35d2a82e59e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-a5153f99-170c-4d49-b9fb-e35d2a82e59e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:41:01.504036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:01.504990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:01.505830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 543/1065] 10.222.0.121 () {56 vars in 1125 bytes} [Thu May 26 17:41:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:41:01.511204 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5153f99-170c-4d49-b9fb-e35d2a82e59e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:01.547554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653eccd6-2b2c-4e67-915c-3640a6382415 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:01.586216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de677fc-e63f-4973-a6b9-ffa1d56baf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:01.586216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de677fc-e63f-4973-a6b9-ffa1d56baf77 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:01.586216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de677fc-e63f-4973-a6b9-ffa1d56baf77 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:01.586216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de677fc-e63f-4973-a6b9-ffa1d56baf77 None admin] PATH_INFO: `/v3/projects/f1eeba4211224510b61e176e58c9d176` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:01.587454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de677fc-e63f-4973-a6b9-ffa1d56baf77 None admin] RBAC: Authorizing `identity:get_project(project_id=f1eeba4211224510b61e176e58c9d176)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:01.588890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de677fc-e63f-4973-a6b9-ffa1d56baf77 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:01.590605 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 523/1066] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:41:01 2022] GET /identity/v3/projects/f1eeba4211224510b61e176e58c9d176 => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:01.597116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de677fc-e63f-4973-a6b9-ffa1d56baf77 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:01.616004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5153f99-170c-4d49-b9fb-e35d2a82e59e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:01.638141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:01.650937 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f3be4-52ea-4e58-9c5d-082f03f69a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:01.651641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f3be4-52ea-4e58-9c5d-082f03f69a8f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:01.651966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f3be4-52ea-4e58-9c5d-082f03f69a8f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:01.652241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f3be4-52ea-4e58-9c5d-082f03f69a8f None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:01.652982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f3be4-52ea-4e58-9c5d-082f03f69a8f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:01.653284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f3be4-52ea-4e58-9c5d-082f03f69a8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:01.654646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f3be4-52ea-4e58-9c5d-082f03f69a8f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:01.660186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:01.664927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 544/1067] 10.222.0.121 () {56 vars in 1120 bytes} [Thu May 26 17:41:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:01.671846 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f3be4-52ea-4e58-9c5d-082f03f69a8f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:01.699160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de677fc-e63f-4973-a6b9-ffa1d56baf77 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:01.732221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cecb6-5187-4bd8-a160-83d274e5217e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:01.732869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cecb6-5187-4bd8-a160-83d274e5217e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:01.733145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cecb6-5187-4bd8-a160-83d274e5217e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:01.733411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cecb6-5187-4bd8-a160-83d274e5217e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:01.734937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cecb6-5187-4bd8-a160-83d274e5217e None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=f1eeba4211224510b61e176e58c9d176, user.id=139081ac5e8546cb959ab468ec59722c) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:01.735602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cecb6-5187-4bd8-a160-83d274e5217e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:01.736601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cecb6-5187-4bd8-a160-83d274e5217e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:01.742411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 524/1068] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:41:01 2022] GET /identity/v3/role_assignments?user.id=139081ac5e8546cb959ab468ec59722c&scope.project.id=f1eeba4211224510b61e176e58c9d176&role.id=4f574961798246a4b12ad2276e67685d => generated 260 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:01.775587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f3be4-52ea-4e58-9c5d-082f03f69a8f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:01.813916 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11935453-f9a7-4a4f-92d8-26d8ab466598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:01.814734 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11935453-f9a7-4a4f-92d8-26d8ab466598 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:01.815041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11935453-f9a7-4a4f-92d8-26d8ab466598 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:01.815310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11935453-f9a7-4a4f-92d8-26d8ab466598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:01.826729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11935453-f9a7-4a4f-92d8-26d8ab466598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:01.828494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11935453-f9a7-4a4f-92d8-26d8ab466598 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-11935453-f9a7-4a4f-92d8-26d8ab466598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:41:01.847526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:01.848843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:01.851381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:41:01.851381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:41:01.851381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:41:01.851381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:01.851381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 545/1069] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:01.854774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cecb6-5187-4bd8-a160-83d274e5217e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:01.965998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cecb6-5187-4bd8-a160-83d274e5217e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:01.991471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:02.000942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554ca1c-0510-4e10-8331-da763014c63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:02.002939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554ca1c-0510-4e10-8331-da763014c63f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:02.002939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554ca1c-0510-4e10-8331-da763014c63f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:02.002939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554ca1c-0510-4e10-8331-da763014c63f None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:02.003198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554ca1c-0510-4e10-8331-da763014c63f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:02.003314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554ca1c-0510-4e10-8331-da763014c63f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:02.004754 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554ca1c-0510-4e10-8331-da763014c63f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:02.016727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 525/1070] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:41:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:02.022219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11935453-f9a7-4a4f-92d8-26d8ab466598 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:02.057039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:02.127737 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11935453-f9a7-4a4f-92d8-26d8ab466598 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:02.161672 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c87b7e-da6a-4ad0-ac06-0e47b9fbfac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:02.162462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c87b7e-da6a-4ad0-ac06-0e47b9fbfac6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:02.162741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c87b7e-da6a-4ad0-ac06-0e47b9fbfac6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:02.163047 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c87b7e-da6a-4ad0-ac06-0e47b9fbfac6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:02.170561 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c87b7e-da6a-4ad0-ac06-0e47b9fbfac6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:02.172068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c87b7e-da6a-4ad0-ac06-0e47b9fbfac6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-d5c87b7e-da6a-4ad0-ac06-0e47b9fbfac6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:41:02.186482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:02.188174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:02.189141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:02.189141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:02.189141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:41:02.189141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:41:02.189141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:41:02.189141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:41:02.189141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:41:02.189141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:41:02.189141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:41:02.189141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:02.189141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 546/1071] 10.222.0.121 () {56 vars in 1098 bytes} [Thu May 26 17:41:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:02.194330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554ca1c-0510-4e10-8331-da763014c63f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:02.295126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554ca1c-0510-4e10-8331-da763014c63f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:02.331403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1ac2b-476c-479f-bad3-189ae241a248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:02.332077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1ac2b-476c-479f-bad3-189ae241a248 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:02.332347 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1ac2b-476c-479f-bad3-189ae241a248 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:02.332623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1ac2b-476c-479f-bad3-189ae241a248 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:02.333456 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1ac2b-476c-479f-bad3-189ae241a248 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:02.333768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1ac2b-476c-479f-bad3-189ae241a248 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:02.335239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1ac2b-476c-479f-bad3-189ae241a248 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:02.353321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 526/1072] 10.222.0.121 () {56 vars in 1138 bytes} [Thu May 26 17:41:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:02.359937 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c87b7e-da6a-4ad0-ac06-0e47b9fbfac6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:02.457149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:02.462894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c87b7e-da6a-4ad0-ac06-0e47b9fbfac6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:02.497124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cfc95-d524-422a-bc94-37ca609efeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:02.497932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cfc95-d524-422a-bc94-37ca609efeeb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:02.498480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cfc95-d524-422a-bc94-37ca609efeeb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:02.498940 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cfc95-d524-422a-bc94-37ca609efeeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:02.509675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cfc95-d524-422a-bc94-37ca609efeeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:02.511149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cfc95-d524-422a-bc94-37ca609efeeb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-176cfc95-d524-422a-bc94-37ca609efeeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:41:02.529088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:02.530060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:02.532109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:41:02.532109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:41:02.532109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:41:02.532109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:02.532109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 547/1073] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:41:02.539217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1ac2b-476c-479f-bad3-189ae241a248 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:02.586712 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:02.639254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1ac2b-476c-479f-bad3-189ae241a248 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:02.673703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d4bec-82d8-4f4d-b8db-e09bca706512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:02.674367 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d4bec-82d8-4f4d-b8db-e09bca706512 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:02.674652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d4bec-82d8-4f4d-b8db-e09bca706512 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:02.674952 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d4bec-82d8-4f4d-b8db-e09bca706512 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:02.675668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d4bec-82d8-4f4d-b8db-e09bca706512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:02.675977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d4bec-82d8-4f4d-b8db-e09bca706512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:02.677336 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d4bec-82d8-4f4d-b8db-e09bca706512 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:02.689663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 527/1074] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:41:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:02.695255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cfc95-d524-422a-bc94-37ca609efeeb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:02.797185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cfc95-d524-422a-bc94-37ca609efeeb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:02.837095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcc155-6739-4532-872c-b57f9a188d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:02.837856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcc155-6739-4532-872c-b57f9a188d2e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:02.838139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcc155-6739-4532-872c-b57f9a188d2e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:02.838415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcc155-6739-4532-872c-b57f9a188d2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:02.846155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fcc155-6739-4532-872c-b57f9a188d2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:02.847689 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fcc155-6739-4532-872c-b57f9a188d2e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-23fcc155-6739-4532-872c-b57f9a188d2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:41:02.863336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:02.864389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:02.865438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:41:02.865438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:41:02.865438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:41:02.865438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:41:02.865438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:41:02.865438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:41:02.865438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:41:02.865438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:02.865438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 548/1075] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:41:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:02.870498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d4bec-82d8-4f4d-b8db-e09bca706512 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:02.975053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d4bec-82d8-4f4d-b8db-e09bca706512 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:02.977416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:03.011304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae38b25-a7da-4964-b2b4-0128b47c19ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:03.011971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae38b25-a7da-4964-b2b4-0128b47c19ce None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:03.012250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae38b25-a7da-4964-b2b4-0128b47c19ce None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:03.012525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae38b25-a7da-4964-b2b4-0128b47c19ce None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:03.013307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae38b25-a7da-4964-b2b4-0128b47c19ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:03.013613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae38b25-a7da-4964-b2b4-0128b47c19ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:03.015151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae38b25-a7da-4964-b2b4-0128b47c19ce None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:03.027088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 528/1076] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:41:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:03.033740 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcc155-6739-4532-872c-b57f9a188d2e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:03.047557 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:03.141821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcc155-6739-4532-872c-b57f9a188d2e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:03.178546 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b359c6-dd06-44ff-89eb-330bfa8b39c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:03.179476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b359c6-dd06-44ff-89eb-330bfa8b39c9 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:03.179759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b359c6-dd06-44ff-89eb-330bfa8b39c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:03.180034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b359c6-dd06-44ff-89eb-330bfa8b39c9 None admin] PATH_INFO: `/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/c0889a69a018452d8de1924072739d17/roles/1808269227c74f569faa53261e45ca40` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:03.182213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b359c6-dd06-44ff-89eb-330bfa8b39c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61dea65b34b47a186151c6a63bc7d79, user_id=c0889a69a018452d8de1924072739d17, role_id=1808269227c74f569faa53261e45ca40)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:03.183691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b359c6-dd06-44ff-89eb-330bfa8b39c9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:03.259462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 549/1077] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:41:03 2022] PUT /identity/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/c0889a69a018452d8de1924072739d17/roles/1808269227c74f569faa53261e45ca40 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:41:03.989073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c459e3-85c4-4295-becd-d11ebef696b5 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:03.989609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c459e3-85c4-4295-becd-d11ebef696b5 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:03.989609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c459e3-85c4-4295-becd-d11ebef696b5 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:03.991160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 529/1078] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:03.996768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fb329-ee29-4d52-9214-641f2333d610 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:03.997271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fb329-ee29-4d52-9214-641f2333d610 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:03.997361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fb329-ee29-4d52-9214-641f2333d610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:04.015885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-410fb329-ee29-4d52-9214-641f2333d610 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:04.017097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-410fb329-ee29-4d52-9214-641f2333d610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:04.095237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:04.098386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 550/1079] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:03 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:04.121746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c459e3-85c4-4295-becd-d11ebef696b5 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:04.224326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c459e3-85c4-4295-becd-d11ebef696b5 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:04.257680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ff9f8-1ba7-4d24-bb2a-abce8b3cb852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:04.258378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ff9f8-1ba7-4d24-bb2a-abce8b3cb852 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:04.258655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ff9f8-1ba7-4d24-bb2a-abce8b3cb852 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:04.258956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ff9f8-1ba7-4d24-bb2a-abce8b3cb852 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:04.268863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ff9f8-1ba7-4d24-bb2a-abce8b3cb852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-1a0ff9f8-1ba7-4d24-bb2a-abce8b3cb852 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:41:04.276308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:04.277378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:41:04.278796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 530/1080] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:41:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:04.283734 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fb329-ee29-4d52-9214-641f2333d610 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:04.381512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fb329-ee29-4d52-9214-641f2333d610 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:04.425665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ca2c2-d514-4f77-8fc7-5513ec11a665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:04.426338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ca2c2-d514-4f77-8fc7-5513ec11a665 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:04.426611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ca2c2-d514-4f77-8fc7-5513ec11a665 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:04.426908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ca2c2-d514-4f77-8fc7-5513ec11a665 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:04.427663 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ca2c2-d514-4f77-8fc7-5513ec11a665 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:04.427963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ca2c2-d514-4f77-8fc7-5513ec11a665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:04.429333 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ca2c2-d514-4f77-8fc7-5513ec11a665 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:04.438285 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 551/1081] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:41:04 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:04.444219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ff9f8-1ba7-4d24-bb2a-abce8b3cb852 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:04.470974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:04.552487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ff9f8-1ba7-4d24-bb2a-abce8b3cb852 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:04.586075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfc2d5-2a4b-426c-8cd8-c51f7220025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:04.586978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfc2d5-2a4b-426c-8cd8-c51f7220025f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:04.587259 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfc2d5-2a4b-426c-8cd8-c51f7220025f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:04.587530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfc2d5-2a4b-426c-8cd8-c51f7220025f None admin] PATH_INFO: `/v3/users/139081ac5e8546cb959ab468ec59722c` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:04.588711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfc2d5-2a4b-426c-8cd8-c51f7220025f None admin] RBAC: Authorizing `identity:get_user(user_id=139081ac5e8546cb959ab468ec59722c)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:04.590171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfc2d5-2a4b-426c-8cd8-c51f7220025f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:04.591848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 531/1082] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:04 2022] GET /identity/v3/users/139081ac5e8546cb959ab468ec59722c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:04.598699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ca2c2-d514-4f77-8fc7-5513ec11a665 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:04.656786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:04.697349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ca2c2-d514-4f77-8fc7-5513ec11a665 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:04.730797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cfe43-f3ff-4dc2-afc9-4431c6a41bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:04.731680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cfe43-f3ff-4dc2-afc9-4431c6a41bde None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:04.731959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cfe43-f3ff-4dc2-afc9-4431c6a41bde None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:04.732230 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cfe43-f3ff-4dc2-afc9-4431c6a41bde None admin] PATH_INFO: `/v3/projects/f1eeba4211224510b61e176e58c9d176` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:04.733451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9cfe43-f3ff-4dc2-afc9-4431c6a41bde None admin] RBAC: Authorizing `identity:get_project(project_id=f1eeba4211224510b61e176e58c9d176)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:04.735122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9cfe43-f3ff-4dc2-afc9-4431c6a41bde None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:04.736820 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 552/1083] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:41:04 2022] GET /identity/v3/projects/f1eeba4211224510b61e176e58c9d176 => generated 312 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:04.742684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfc2d5-2a4b-426c-8cd8-c51f7220025f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:04.832848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:04.841436 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfc2d5-2a4b-426c-8cd8-c51f7220025f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:04.877679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e3e8f-8aea-4299-ab50-c0634b5ab874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:04.878571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e3e8f-8aea-4299-ab50-c0634b5ab874 None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:04.878869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e3e8f-8aea-4299-ab50-c0634b5ab874 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:04.879144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e3e8f-8aea-4299-ab50-c0634b5ab874 None admin] PATH_INFO: `/v3/projects/f1eeba4211224510b61e176e58c9d176/users/139081ac5e8546cb959ab468ec59722c/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:04.881063 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e3e8f-8aea-4299-ab50-c0634b5ab874 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1eeba4211224510b61e176e58c9d176, user_id=139081ac5e8546cb959ab468ec59722c, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:04.882475 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e3e8f-8aea-4299-ab50-c0634b5ab874 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:04.950549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 532/1084] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:41:04 2022] PUT /identity/v3/projects/f1eeba4211224510b61e176e58c9d176/users/139081ac5e8546cb959ab468ec59722c/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:41:05.515135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda90e8-3189-499f-bb7a-1b0e2c04c9e0 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:05.515699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda90e8-3189-499f-bb7a-1b0e2c04c9e0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:05.515699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda90e8-3189-499f-bb7a-1b0e2c04c9e0 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:05.517033 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 553/1085] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:05 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:05.522586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86335fc-988b-44e3-b01e-8b84a5bc359e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:05.523034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86335fc-988b-44e3-b01e-8b84a5bc359e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:05.523170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86335fc-988b-44e3-b01e-8b84a5bc359e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:05.540923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d86335fc-988b-44e3-b01e-8b84a5bc359e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:05.542143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d86335fc-988b-44e3-b01e-8b84a5bc359e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:05.610607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 533/1086] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:05 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:05.634000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda90e8-3189-499f-bb7a-1b0e2c04c9e0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:05.737558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda90e8-3189-499f-bb7a-1b0e2c04c9e0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:05.767750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:05.776440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbda81-3265-48cb-8c93-47daf489a251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:05.777255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbda81-3265-48cb-8c93-47daf489a251 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:05.777624 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbda81-3265-48cb-8c93-47daf489a251 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:05.777822 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbda81-3265-48cb-8c93-47daf489a251 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:05.788447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efbda81-3265-48cb-8c93-47daf489a251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-0efbda81-3265-48cb-8c93-47daf489a251 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:41:05.797302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:05.798410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:05.799297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 554/1087] 10.222.0.121 () {56 vars in 1125 bytes} [Thu May 26 17:41:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:05.804599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86335fc-988b-44e3-b01e-8b84a5bc359e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:05.884604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:05.907566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86335fc-988b-44e3-b01e-8b84a5bc359e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:05.944212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4f55b-4330-4fa4-abd8-115df68876ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:05.944886 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4f55b-4330-4fa4-abd8-115df68876ef None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:05.945167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4f55b-4330-4fa4-abd8-115df68876ef None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:05.945440 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4f55b-4330-4fa4-abd8-115df68876ef None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:05.946251 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4f55b-4330-4fa4-abd8-115df68876ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:05.946831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4f55b-4330-4fa4-abd8-115df68876ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:05.948538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4f55b-4330-4fa4-abd8-115df68876ef None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:05.956754 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 534/1088] 10.222.0.121 () {56 vars in 1120 bytes} [Thu May 26 17:41:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:05.962375 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbda81-3265-48cb-8c93-47daf489a251 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:06.065279 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbda81-3265-48cb-8c93-47daf489a251 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:06.100394 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d100ad-a8cf-4edf-866d-7ca7c10fa9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:06.101191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d100ad-a8cf-4edf-866d-7ca7c10fa9bf None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:06.101467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d100ad-a8cf-4edf-866d-7ca7c10fa9bf None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:06.101738 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d100ad-a8cf-4edf-866d-7ca7c10fa9bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:06.114168 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d100ad-a8cf-4edf-866d-7ca7c10fa9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:06.115675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d100ad-a8cf-4edf-866d-7ca7c10fa9bf None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-13d100ad-a8cf-4edf-866d-7ca7c10fa9bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:41:06.135288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:06.136377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:06.137313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:41:06.137313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:41:06.137313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:41:06.137313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:06.137313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 555/1089] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:41:06.143526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4f55b-4330-4fa4-abd8-115df68876ef None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:06.244295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4f55b-4330-4fa4-abd8-115df68876ef None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:06.277572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdadde4-ba86-4df1-9122-2c96da86ac6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:06.278235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdadde4-ba86-4df1-9122-2c96da86ac6e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:06.278519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdadde4-ba86-4df1-9122-2c96da86ac6e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:06.278792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdadde4-ba86-4df1-9122-2c96da86ac6e None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:06.279534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdadde4-ba86-4df1-9122-2c96da86ac6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:06.279839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdadde4-ba86-4df1-9122-2c96da86ac6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:06.281263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdadde4-ba86-4df1-9122-2c96da86ac6e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:06.293061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 535/1090] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:41:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:06.299903 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d100ad-a8cf-4edf-866d-7ca7c10fa9bf None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:06.341281 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:06.397503 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d100ad-a8cf-4edf-866d-7ca7c10fa9bf None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:06.430725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b4e99-713c-46c2-8df4-7742e768f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:06.431544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b4e99-713c-46c2-8df4-7742e768f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:06.431821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b4e99-713c-46c2-8df4-7742e768f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:06.432093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b4e99-713c-46c2-8df4-7742e768f4c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:06.440149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b4e99-713c-46c2-8df4-7742e768f4c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:06.441577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b4e99-713c-46c2-8df4-7742e768f4c4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-886b4e99-713c-46c2-8df4-7742e768f4c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:41:06.455164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:06.456278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:06.457470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:06.457470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:06.457470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:41:06.457470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:41:06.457470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:41:06.457470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:41:06.457470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:41:06.457470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:41:06.457470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:41:06.457470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:06.458961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 556/1091] 10.222.0.121 () {56 vars in 1098 bytes} [Thu May 26 17:41:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:06.462038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdadde4-ba86-4df1-9122-2c96da86ac6e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:06.465845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:06.562474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdadde4-ba86-4df1-9122-2c96da86ac6e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:06.597029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb277c-a2f9-44e1-9e79-947df47473f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:06.597618 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb277c-a2f9-44e1-9e79-947df47473f0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:06.597894 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb277c-a2f9-44e1-9e79-947df47473f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:06.598162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb277c-a2f9-44e1-9e79-947df47473f0 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:06.599085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb277c-a2f9-44e1-9e79-947df47473f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:06.599396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb277c-a2f9-44e1-9e79-947df47473f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:06.600799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb277c-a2f9-44e1-9e79-947df47473f0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:06.619061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 536/1092] 10.222.0.121 () {56 vars in 1138 bytes} [Thu May 26 17:41:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:06.625768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b4e99-713c-46c2-8df4-7742e768f4c4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:06.722430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:06.725619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b4e99-713c-46c2-8df4-7742e768f4c4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:06.759850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2095d-0376-4ca9-aafd-0588a889f59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:06.760493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2095d-0376-4ca9-aafd-0588a889f59b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:06.760771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2095d-0376-4ca9-aafd-0588a889f59b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:06.761042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2095d-0376-4ca9-aafd-0588a889f59b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:06.828578 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2095d-0376-4ca9-aafd-0588a889f59b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:06.830018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2095d-0376-4ca9-aafd-0588a889f59b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:06.926139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:06.930024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 557/1093] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:41:06 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 17:41:06.936244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb277c-a2f9-44e1-9e79-947df47473f0 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:07.001844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:07.042473 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb277c-a2f9-44e1-9e79-947df47473f0 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:07.075787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382dc4cc-ec09-411c-abd2-e166387f75e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:07.076578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382dc4cc-ec09-411c-abd2-e166387f75e1 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:07.076858 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382dc4cc-ec09-411c-abd2-e166387f75e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:07.078011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382dc4cc-ec09-411c-abd2-e166387f75e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:07.088382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382dc4cc-ec09-411c-abd2-e166387f75e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:07.089800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382dc4cc-ec09-411c-abd2-e166387f75e1 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-382dc4cc-ec09-411c-abd2-e166387f75e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:41:07.107941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:07.109547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:07.110443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:41:07.110443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:41:07.110443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:41:07.110443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:41:07.110443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 537/1094] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:07.115577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2095d-0376-4ca9-aafd-0588a889f59b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:07.174270 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ceaec-1491-41ed-a902-626fe2eb4fe8 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:07.174550 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ceaec-1491-41ed-a902-626fe2eb4fe8 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:07.174844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ceaec-1491-41ed-a902-626fe2eb4fe8 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:07.176289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 538/1095] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:41:07.181194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c9594-f63f-4c97-95dd-5eecabf5de11 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:07.181477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c9594-f63f-4c97-95dd-5eecabf5de11 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:07.181755 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c9594-f63f-4c97-95dd-5eecabf5de11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:07.199553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-9f4c9594-f63f-4c97-95dd-5eecabf5de11 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:07.200744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9f4c9594-f63f-4c97-95dd-5eecabf5de11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:07.221357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2095d-0376-4ca9-aafd-0588a889f59b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:07.257030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d81ca-ef8a-46e4-87ac-a8bb1dbc5007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:07.257696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d81ca-ef8a-46e4-87ac-a8bb1dbc5007 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:07.257979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d81ca-ef8a-46e4-87ac-a8bb1dbc5007 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:07.258251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d81ca-ef8a-46e4-87ac-a8bb1dbc5007 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:07.259002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d81ca-ef8a-46e4-87ac-a8bb1dbc5007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:07.259306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d81ca-ef8a-46e4-87ac-a8bb1dbc5007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:07.260669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d81ca-ef8a-46e4-87ac-a8bb1dbc5007 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:07.270505 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 539/1096] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:07 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:41:07.276974 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 558/1097] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:41:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:07.278049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c9594-f63f-4c97-95dd-5eecabf5de11 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:07.293310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d81ca-ef8a-46e4-87ac-a8bb1dbc5007 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:07.324799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:07.352527 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:07.379681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c9594-f63f-4c97-95dd-5eecabf5de11 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:07.393318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d81ca-ef8a-46e4-87ac-a8bb1dbc5007 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:07.413224 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1f7ec-cd73-4bd2-889a-46b2192d95ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:07.414016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1f7ec-cd73-4bd2-889a-46b2192d95ab None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:07.414291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1f7ec-cd73-4bd2-889a-46b2192d95ab None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:07.414562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1f7ec-cd73-4bd2-889a-46b2192d95ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:07.422334 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1f7ec-cd73-4bd2-889a-46b2192d95ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:07.423807 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1f7ec-cd73-4bd2-889a-46b2192d95ab None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:07.428869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc60f52-e1a7-43fb-9caf-e1f3bd05cba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:07.429609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc60f52-e1a7-43fb-9caf-e1f3bd05cba5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:07.429894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc60f52-e1a7-43fb-9caf-e1f3bd05cba5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:07.430173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc60f52-e1a7-43fb-9caf-e1f3bd05cba5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-76c1f7ec-cd73-4bd2-889a-46b2192d95ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:41:07.438549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:07.439652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:07.440571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:41:07.440571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:41:07.440571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:41:07.440571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:41:07.440571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:41:07.440571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:41:07.440571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:41:07.440571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:41:07.440571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 540/1098] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:41:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:07.441060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc60f52-e1a7-43fb-9caf-e1f3bd05cba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:41:07.445655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1f7ec-cd73-4bd2-889a-46b2192d95ab None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-bdc60f52-e1a7-43fb-9caf-e1f3bd05cba5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:41:07.448291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:07.449321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:07.450183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 559/1099] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:41:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:41:07.455790 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc60f52-e1a7-43fb-9caf-e1f3bd05cba5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:07.544866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1f7ec-cd73-4bd2-889a-46b2192d95ab None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:07.547467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:07.556736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc60f52-e1a7-43fb-9caf-e1f3bd05cba5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:07.578070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:07.580565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b798c08-db2a-43ca-a9a6-1001c651f17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:07.581434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b798c08-db2a-43ca-a9a6-1001c651f17f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:07.581951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b798c08-db2a-43ca-a9a6-1001c651f17f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:07.582536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b798c08-db2a-43ca-a9a6-1001c651f17f None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:07.583366 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b798c08-db2a-43ca-a9a6-1001c651f17f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:07.583678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b798c08-db2a-43ca-a9a6-1001c651f17f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:07.585122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b798c08-db2a-43ca-a9a6-1001c651f17f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:07.593146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5984e-be43-4c09-9f75-7600b9f1de61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:07.594101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5984e-be43-4c09-9f75-7600b9f1de61 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:07.594398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5984e-be43-4c09-9f75-7600b9f1de61 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:07.594676 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5984e-be43-4c09-9f75-7600b9f1de61 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:07.595449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5984e-be43-4c09-9f75-7600b9f1de61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:07.595759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5984e-be43-4c09-9f75-7600b9f1de61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:07.597126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5984e-be43-4c09-9f75-7600b9f1de61 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:07.597822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 541/1100] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:07.603585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b798c08-db2a-43ca-a9a6-1001c651f17f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:07.612091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 560/1101] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:41:07 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:07.613171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5984e-be43-4c09-9f75-7600b9f1de61 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:07.710155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b798c08-db2a-43ca-a9a6-1001c651f17f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:07.718421 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5984e-be43-4c09-9f75-7600b9f1de61 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:07.744833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ac14f-4ba9-4cac-94e2-30954ccfc234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:07.745497 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ac14f-4ba9-4cac-94e2-30954ccfc234 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:07.745780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ac14f-4ba9-4cac-94e2-30954ccfc234 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:07.746054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ac14f-4ba9-4cac-94e2-30954ccfc234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:07.746928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ac14f-4ba9-4cac-94e2-30954ccfc234 None admin] RBAC: Adding query filter params (role.id=1808269227c74f569faa53261e45ca40, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=c0889a69a018452d8de1924072739d17) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:07.747243 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ac14f-4ba9-4cac-94e2-30954ccfc234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:07.748679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ac14f-4ba9-4cac-94e2-30954ccfc234 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:07.755081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 542/1102] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:41:07 2022] GET /identity/v3/role_assignments?user.id=c0889a69a018452d8de1924072739d17&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=1808269227c74f569faa53261e45ca40 => generated 610 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:07.762452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252de56-d96d-4a25-98b7-cb0751e8c7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:07.763477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252de56-d96d-4a25-98b7-cb0751e8c7ea None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:07.763761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252de56-d96d-4a25-98b7-cb0751e8c7ea None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:07.764037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252de56-d96d-4a25-98b7-cb0751e8c7ea None admin] PATH_INFO: `/v3/users/139081ac5e8546cb959ab468ec59722c` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:07.766437 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252de56-d96d-4a25-98b7-cb0751e8c7ea None admin] RBAC: Authorizing `identity:get_user(user_id=139081ac5e8546cb959ab468ec59722c)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:07.768059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252de56-d96d-4a25-98b7-cb0751e8c7ea None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:07.770369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 561/1103] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:07 2022] GET /identity/v3/users/139081ac5e8546cb959ab468ec59722c => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:07.776918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ac14f-4ba9-4cac-94e2-30954ccfc234 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:07.809945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:07.895298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ac14f-4ba9-4cac-94e2-30954ccfc234 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:07.938893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be04735-5364-44fe-8e07-d3c548a64d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:07.939561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be04735-5364-44fe-8e07-d3c548a64d8d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:07.939841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be04735-5364-44fe-8e07-d3c548a64d8d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:07.940113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be04735-5364-44fe-8e07-d3c548a64d8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:08.019257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be04735-5364-44fe-8e07-d3c548a64d8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:08.020633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be04735-5364-44fe-8e07-d3c548a64d8d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:08.050152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:08.129453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 543/1104] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:41:07 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:41:08.135579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252de56-d96d-4a25-98b7-cb0751e8c7ea None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:08.190434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:08.252736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252de56-d96d-4a25-98b7-cb0751e8c7ea None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:08.295816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d12b4-9ced-4b29-85be-3a77e1999133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:08.297539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d12b4-9ced-4b29-85be-3a77e1999133 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:08.297753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d12b4-9ced-4b29-85be-3a77e1999133 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:08.298026 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d12b4-9ced-4b29-85be-3a77e1999133 None admin] PATH_INFO: `/v3/projects/f1eeba4211224510b61e176e58c9d176` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:08.299339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d12b4-9ced-4b29-85be-3a77e1999133 None admin] RBAC: Authorizing `identity:get_project(project_id=f1eeba4211224510b61e176e58c9d176)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:08.300781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d12b4-9ced-4b29-85be-3a77e1999133 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:08.302494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 562/1105] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:41:08 2022] GET /identity/v3/projects/f1eeba4211224510b61e176e58c9d176 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:08.308807 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be04735-5364-44fe-8e07-d3c548a64d8d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:08.421669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be04735-5364-44fe-8e07-d3c548a64d8d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:08.448689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:08.457411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2397e0-b307-4266-bd34-41da9cc433bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:08.458082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2397e0-b307-4266-bd34-41da9cc433bf None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:08.462837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2397e0-b307-4266-bd34-41da9cc433bf None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:08.462837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2397e0-b307-4266-bd34-41da9cc433bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:08.462837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2397e0-b307-4266-bd34-41da9cc433bf None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=f1eeba4211224510b61e176e58c9d176, user.id=139081ac5e8546cb959ab468ec59722c) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:08.462837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2397e0-b307-4266-bd34-41da9cc433bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:08.462837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2397e0-b307-4266-bd34-41da9cc433bf None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:08.467663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 544/1106] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:41:08 2022] GET /identity/v3/role_assignments?user.id=139081ac5e8546cb959ab468ec59722c&scope.project.id=f1eeba4211224510b61e176e58c9d176&role.id=4f574961798246a4b12ad2276e67685d => generated 610 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:10.726849 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27975dc-4a56-44b8-ae50-796185b2df07 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:10.727390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27975dc-4a56-44b8-ae50-796185b2df07 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:10.727390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27975dc-4a56-44b8-ae50-796185b2df07 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:10.728767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 563/1107] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:10.735145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e985ec-0e00-4f46-bdc5-07fd5f4987ea None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:10.735696 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e985ec-0e00-4f46-bdc5-07fd5f4987ea None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:10.735696 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e985ec-0e00-4f46-bdc5-07fd5f4987ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:10.753692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a3e985ec-0e00-4f46-bdc5-07fd5f4987ea None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:10.754948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a3e985ec-0e00-4f46-bdc5-07fd5f4987ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:10.824897 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 545/1108] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:10 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:10.850575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27975dc-4a56-44b8-ae50-796185b2df07 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:10.973759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27975dc-4a56-44b8-ae50-796185b2df07 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:10.992818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:11.010328 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ae0ef-9557-4988-88dc-6ebecf9d916c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:11.011151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ae0ef-9557-4988-88dc-6ebecf9d916c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:11.011506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ae0ef-9557-4988-88dc-6ebecf9d916c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:11.011778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ae0ef-9557-4988-88dc-6ebecf9d916c None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:11.012940 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ae0ef-9557-4988-88dc-6ebecf9d916c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:11.014396 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ae0ef-9557-4988-88dc-6ebecf9d916c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:11.016047 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 564/1109] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:41:10 2022] GET /identity/v3/domains/default => generated 200 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:11.021968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e985ec-0e00-4f46-bdc5-07fd5f4987ea None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:11.129403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e985ec-0e00-4f46-bdc5-07fd5f4987ea None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:11.158357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:11.171169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ac661-2207-4b3f-ae81-41704fdc0747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:11.172178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ac661-2207-4b3f-ae81-41704fdc0747 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:11.172458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ac661-2207-4b3f-ae81-41704fdc0747 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:11.172730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ac661-2207-4b3f-ae81-41704fdc0747 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:11.173452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ac661-2207-4b3f-ae81-41704fdc0747 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:11.174919 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ac661-2207-4b3f-ae81-41704fdc0747 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:11.275812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 546/1110] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:41:11 2022] POST /identity/v3/users => generated 287 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:41:13.517132 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b14c8a-7d39-447e-88a3-320679421aa2 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:13.517697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b14c8a-7d39-447e-88a3-320679421aa2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:13.517697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b14c8a-7d39-447e-88a3-320679421aa2 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:13.519079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 565/1111] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:13.524247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a013bc4-45be-4eb3-be57-c74c063531a6 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:13.524641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a013bc4-45be-4eb3-be57-c74c063531a6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:13.524807 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a013bc4-45be-4eb3-be57-c74c063531a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:13.541415 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-2a013bc4-45be-4eb3-be57-c74c063531a6 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:13.542583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-2a013bc4-45be-4eb3-be57-c74c063531a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:13.609642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 547/1112] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:13 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:13.632657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b14c8a-7d39-447e-88a3-320679421aa2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:13.736751 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b14c8a-7d39-447e-88a3-320679421aa2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:13.771602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d1da5-5a2e-4c4d-9408-6fe8ea871834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:13.772394 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d1da5-5a2e-4c4d-9408-6fe8ea871834 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:13.772680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d1da5-5a2e-4c4d-9408-6fe8ea871834 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:13.772956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d1da5-5a2e-4c4d-9408-6fe8ea871834 None admin] PATH_INFO: `/v3/roles/582f59fc05ce474287c6fdc503f9ac82` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:13.774419 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d1da5-5a2e-4c4d-9408-6fe8ea871834 None admin] RBAC: Authorizing `identity:get_role(role_id=582f59fc05ce474287c6fdc503f9ac82)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:13.775816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d1da5-5a2e-4c4d-9408-6fe8ea871834 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:13.777187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 566/1113] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:13 2022] GET /identity/v3/roles/582f59fc05ce474287c6fdc503f9ac82 => generated 224 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:13.783690 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a013bc4-45be-4eb3-be57-c74c063531a6 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:13.884589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a013bc4-45be-4eb3-be57-c74c063531a6 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:13.894260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:13.918995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89cda7e-30db-49ac-86eb-b5c63a0c0e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:13.919925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cda7e-30db-49ac-86eb-b5c63a0c0e66 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:13.920510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cda7e-30db-49ac-86eb-b5c63a0c0e66 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:13.920971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cda7e-30db-49ac-86eb-b5c63a0c0e66 None admin] PATH_INFO: `/v3/users/3e47c64c26084621805a53fe62316b3e` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:13.934343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89cda7e-30db-49ac-86eb-b5c63a0c0e66 None admin] RBAC: Authorizing `identity:get_user(user_id=3e47c64c26084621805a53fe62316b3e)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:13.936669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89cda7e-30db-49ac-86eb-b5c63a0c0e66 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:13.938515 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 548/1114] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:13 2022] GET /identity/v3/users/3e47c64c26084621805a53fe62316b3e => generated 287 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:13.944449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d1da5-5a2e-4c4d-9408-6fe8ea871834 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:13.999491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:14.044225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d1da5-5a2e-4c4d-9408-6fe8ea871834 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:14.078337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccb1b2-536b-4ec1-893c-79295110d3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:14.079147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccb1b2-536b-4ec1-893c-79295110d3dc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:14.079425 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccb1b2-536b-4ec1-893c-79295110d3dc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:14.079695 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccb1b2-536b-4ec1-893c-79295110d3dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:14.148986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccb1b2-536b-4ec1-893c-79295110d3dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:14.150436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccb1b2-536b-4ec1-893c-79295110d3dc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:14.203149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:14.248423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 567/1115] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:41:13 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:41:14.254734 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89cda7e-30db-49ac-86eb-b5c63a0c0e66 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:14.351054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89cda7e-30db-49ac-86eb-b5c63a0c0e66 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:14.383702 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff361a-5f98-46e2-97d9-325d5376e590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:14.384540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff361a-5f98-46e2-97d9-325d5376e590 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:14.384818 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff361a-5f98-46e2-97d9-325d5376e590 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:14.385085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff361a-5f98-46e2-97d9-325d5376e590 None admin] PATH_INFO: `/v3/projects/f1eeba4211224510b61e176e58c9d176` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:14.386218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff361a-5f98-46e2-97d9-325d5376e590 None admin] RBAC: Authorizing `identity:get_project(project_id=f1eeba4211224510b61e176e58c9d176)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:14.387644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff361a-5f98-46e2-97d9-325d5376e590 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:14.389511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 549/1116] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:41:14 2022] GET /identity/v3/projects/f1eeba4211224510b61e176e58c9d176 => generated 312 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:14.395875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccb1b2-536b-4ec1-893c-79295110d3dc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:14.493461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccb1b2-536b-4ec1-893c-79295110d3dc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:14.531004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e31d8-a640-47a2-aaad-8a88ad188a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:14.531882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e31d8-a640-47a2-aaad-8a88ad188a5e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:14.532261 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e31d8-a640-47a2-aaad-8a88ad188a5e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:14.532665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e31d8-a640-47a2-aaad-8a88ad188a5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:14.533886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e31d8-a640-47a2-aaad-8a88ad188a5e None admin] RBAC: Adding query filter params (role.id=582f59fc05ce474287c6fdc503f9ac82, scope.project.id=f1eeba4211224510b61e176e58c9d176, user.id=3e47c64c26084621805a53fe62316b3e) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:14.534195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e31d8-a640-47a2-aaad-8a88ad188a5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:14.535652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e31d8-a640-47a2-aaad-8a88ad188a5e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:14.542447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 568/1117] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:41:14 2022] GET /identity/v3/role_assignments?user.id=3e47c64c26084621805a53fe62316b3e&scope.project.id=f1eeba4211224510b61e176e58c9d176&role.id=582f59fc05ce474287c6fdc503f9ac82 => generated 260 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:16.757191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d58bf-3ede-4ec3-8e76-7e2f74901455 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:16.757745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d58bf-3ede-4ec3-8e76-7e2f74901455 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:16.757745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d58bf-3ede-4ec3-8e76-7e2f74901455 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:16.759167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 550/1118] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:16.764989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d0551-cdaa-4705-8602-996b435f91b2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:16.765521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d0551-cdaa-4705-8602-996b435f91b2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:16.765707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d0551-cdaa-4705-8602-996b435f91b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:16.782984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9b5d0551-cdaa-4705-8602-996b435f91b2 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:16.784179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9b5d0551-cdaa-4705-8602-996b435f91b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:16.798970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:16.851692 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 569/1119] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:16 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:16.875011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d58bf-3ede-4ec3-8e76-7e2f74901455 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:16.909495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:16.974700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d58bf-3ede-4ec3-8e76-7e2f74901455 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:17.008272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa530c-c753-4411-911f-6e274faefe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:17.009052 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa530c-c753-4411-911f-6e274faefe5b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:17.009329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa530c-c753-4411-911f-6e274faefe5b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:17.009601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa530c-c753-4411-911f-6e274faefe5b None admin] PATH_INFO: `/v3/roles/582f59fc05ce474287c6fdc503f9ac82` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:17.011078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa530c-c753-4411-911f-6e274faefe5b None admin] RBAC: Authorizing `identity:get_role(role_id=582f59fc05ce474287c6fdc503f9ac82)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:17.012425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa530c-c753-4411-911f-6e274faefe5b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:17.013801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 551/1120] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:16 2022] GET /identity/v3/roles/582f59fc05ce474287c6fdc503f9ac82 => generated 224 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:17.019831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d0551-cdaa-4705-8602-996b435f91b2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:17.117740 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d0551-cdaa-4705-8602-996b435f91b2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:17.151741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbb2ef-c113-4e3a-b7f1-eee4d80b3748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:17.152614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbb2ef-c113-4e3a-b7f1-eee4d80b3748 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:17.152895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbb2ef-c113-4e3a-b7f1-eee4d80b3748 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:17.153170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbb2ef-c113-4e3a-b7f1-eee4d80b3748 None admin] PATH_INFO: `/v3/users/3e47c64c26084621805a53fe62316b3e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:17.154890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbb2ef-c113-4e3a-b7f1-eee4d80b3748 None admin] RBAC: Authorizing `identity:get_user(user_id=3e47c64c26084621805a53fe62316b3e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:17.156773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbb2ef-c113-4e3a-b7f1-eee4d80b3748 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:17.158405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 570/1121] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:17 2022] GET /identity/v3/users/3e47c64c26084621805a53fe62316b3e => generated 287 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:17.164361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa530c-c753-4411-911f-6e274faefe5b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:17.268407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa530c-c753-4411-911f-6e274faefe5b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:17.269862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:17.303704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd10c26-d345-41c1-beb5-132282ea5f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:17.304553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10c26-d345-41c1-beb5-132282ea5f2e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:17.304959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10c26-d345-41c1-beb5-132282ea5f2e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:17.305548 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10c26-d345-41c1-beb5-132282ea5f2e None admin] PATH_INFO: `/v3/projects/f1eeba4211224510b61e176e58c9d176` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:17.306733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd10c26-d345-41c1-beb5-132282ea5f2e None admin] RBAC: Authorizing `identity:get_project(project_id=f1eeba4211224510b61e176e58c9d176)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:17.308190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd10c26-d345-41c1-beb5-132282ea5f2e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:17.309906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 552/1122] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:41:17 2022] GET /identity/v3/projects/f1eeba4211224510b61e176e58c9d176 => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:17.316670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbb2ef-c113-4e3a-b7f1-eee4d80b3748 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:17.335039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:17.416066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbb2ef-c113-4e3a-b7f1-eee4d80b3748 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:17.449633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffdf47c-1921-4188-997c-8efa9c3b60d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:17.450506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdf47c-1921-4188-997c-8efa9c3b60d2 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:17.450783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdf47c-1921-4188-997c-8efa9c3b60d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:17.451087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdf47c-1921-4188-997c-8efa9c3b60d2 None admin] PATH_INFO: `/v3/projects/f1eeba4211224510b61e176e58c9d176/users/3e47c64c26084621805a53fe62316b3e/roles/582f59fc05ce474287c6fdc503f9ac82` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:17.453259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffdf47c-1921-4188-997c-8efa9c3b60d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1eeba4211224510b61e176e58c9d176, user_id=3e47c64c26084621805a53fe62316b3e, role_id=582f59fc05ce474287c6fdc503f9ac82)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:17.454675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffdf47c-1921-4188-997c-8efa9c3b60d2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:17.538319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 571/1123] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:41:17 2022] PUT /identity/v3/projects/f1eeba4211224510b61e176e58c9d176/users/3e47c64c26084621805a53fe62316b3e/roles/582f59fc05ce474287c6fdc503f9ac82 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:41:19.723560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7f7b9-5100-41f5-9913-6733ee6dfb3a None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:19.724106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7f7b9-5100-41f5-9913-6733ee6dfb3a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:19.724106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7f7b9-5100-41f5-9913-6733ee6dfb3a None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:19.725492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 553/1124] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:19.730790 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08934114-6e43-4202-b643-9cb2e8ff9731 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:19.731475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08934114-6e43-4202-b643-9cb2e8ff9731 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:19.731475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08934114-6e43-4202-b643-9cb2e8ff9731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:19.748372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-08934114-6e43-4202-b643-9cb2e8ff9731 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:19.749564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-08934114-6e43-4202-b643-9cb2e8ff9731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:19.816506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 572/1125] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:19 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:19.839597 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7f7b9-5100-41f5-9913-6733ee6dfb3a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:19.941394 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7f7b9-5100-41f5-9913-6733ee6dfb3a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:19.979258 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92524f8-1964-4191-a4db-04c0dedd2223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:19.980662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92524f8-1964-4191-a4db-04c0dedd2223 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:19.981155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92524f8-1964-4191-a4db-04c0dedd2223 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:19.981641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92524f8-1964-4191-a4db-04c0dedd2223 None admin] PATH_INFO: `/v3/roles/582f59fc05ce474287c6fdc503f9ac82` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:19.984712 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92524f8-1964-4191-a4db-04c0dedd2223 None admin] RBAC: Authorizing `identity:get_role(role_id=582f59fc05ce474287c6fdc503f9ac82)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:19.987241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92524f8-1964-4191-a4db-04c0dedd2223 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:19.989934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 554/1126] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:19 2022] GET /identity/v3/roles/582f59fc05ce474287c6fdc503f9ac82 => generated 224 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:19.995668 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08934114-6e43-4202-b643-9cb2e8ff9731 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:20.004929 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:20.103528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08934114-6e43-4202-b643-9cb2e8ff9731 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:20.137488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23080a4-43cf-4fa1-b51f-0fc4bcbe0e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:20.138360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23080a4-43cf-4fa1-b51f-0fc4bcbe0e75 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:20.138784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23080a4-43cf-4fa1-b51f-0fc4bcbe0e75 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:20.139180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23080a4-43cf-4fa1-b51f-0fc4bcbe0e75 None admin] PATH_INFO: `/v3/users/3e47c64c26084621805a53fe62316b3e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:20.140357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23080a4-43cf-4fa1-b51f-0fc4bcbe0e75 None admin] RBAC: Authorizing `identity:get_user(user_id=3e47c64c26084621805a53fe62316b3e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:20.141787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23080a4-43cf-4fa1-b51f-0fc4bcbe0e75 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:20.143459 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 573/1127] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:19 2022] GET /identity/v3/users/3e47c64c26084621805a53fe62316b3e => generated 287 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:20.149434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92524f8-1964-4191-a4db-04c0dedd2223 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:20.190985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:20.249210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92524f8-1964-4191-a4db-04c0dedd2223 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:20.283653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe5872-3e1c-4f11-be26-390d4b46d121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:20.284301 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe5872-3e1c-4f11-be26-390d4b46d121 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:20.284582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe5872-3e1c-4f11-be26-390d4b46d121 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:20.284858 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe5872-3e1c-4f11-be26-390d4b46d121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:20.351069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe5872-3e1c-4f11-be26-390d4b46d121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:20.352504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe5872-3e1c-4f11-be26-390d4b46d121 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:20.390573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:20.449306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 555/1128] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:41:20 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:41:20.455665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23080a4-43cf-4fa1-b51f-0fc4bcbe0e75 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:20.527817 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:20.552477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23080a4-43cf-4fa1-b51f-0fc4bcbe0e75 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:20.585847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb771f7-5b3a-4687-b8e2-ea2aaab4bbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:20.586966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb771f7-5b3a-4687-b8e2-ea2aaab4bbf9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:20.587102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb771f7-5b3a-4687-b8e2-ea2aaab4bbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:20.587340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb771f7-5b3a-4687-b8e2-ea2aaab4bbf9 None admin] PATH_INFO: `/v3/projects/f1eeba4211224510b61e176e58c9d176` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:20.588519 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb771f7-5b3a-4687-b8e2-ea2aaab4bbf9 None admin] RBAC: Authorizing `identity:get_project(project_id=f1eeba4211224510b61e176e58c9d176)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:20.589942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb771f7-5b3a-4687-b8e2-ea2aaab4bbf9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:20.591809 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 574/1129] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:41:20 2022] GET /identity/v3/projects/f1eeba4211224510b61e176e58c9d176 => generated 312 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:20.597990 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe5872-3e1c-4f11-be26-390d4b46d121 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:20.696333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe5872-3e1c-4f11-be26-390d4b46d121 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:20.729845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0e725-db90-42b6-a957-7ceba9a15edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:20.730502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0e725-db90-42b6-a957-7ceba9a15edb None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:20.730751 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0e725-db90-42b6-a957-7ceba9a15edb None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:20.731108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0e725-db90-42b6-a957-7ceba9a15edb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:20.731970 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0e725-db90-42b6-a957-7ceba9a15edb None admin] RBAC: Adding query filter params (role.id=582f59fc05ce474287c6fdc503f9ac82, scope.project.id=f1eeba4211224510b61e176e58c9d176, user.id=3e47c64c26084621805a53fe62316b3e) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:20.732914 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0e725-db90-42b6-a957-7ceba9a15edb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:20.734489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0e725-db90-42b6-a957-7ceba9a15edb None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:20.740334 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 556/1130] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:41:20 2022] GET /identity/v3/role_assignments?user.id=3e47c64c26084621805a53fe62316b3e&scope.project.id=f1eeba4211224510b61e176e58c9d176&role.id=582f59fc05ce474287c6fdc503f9ac82 => generated 610 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:22.960899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb7ab4-088f-4b7c-a072-115da1731859 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:22.961477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb7ab4-088f-4b7c-a072-115da1731859 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:22.961477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb7ab4-088f-4b7c-a072-115da1731859 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:22.962832 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 575/1131] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:22.968394 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462bbe1-e95b-4ed5-be89-42c7ab953a8e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:22.968827 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462bbe1-e95b-4ed5-be89-42c7ab953a8e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:22.968944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462bbe1-e95b-4ed5-be89-42c7ab953a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:22.973902 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:22.986218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5462bbe1-e95b-4ed5-be89-42c7ab953a8e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:22.987414 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5462bbe1-e95b-4ed5-be89-42c7ab953a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:23.054322 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 557/1132] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:22 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:23.078301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb7ab4-088f-4b7c-a072-115da1731859 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:23.184025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb7ab4-088f-4b7c-a072-115da1731859 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:23.219745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b376b-1daa-4ca0-bd83-16a87a780876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:23.220450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b376b-1daa-4ca0-bd83-16a87a780876 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:23.220726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b376b-1daa-4ca0-bd83-16a87a780876 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:23.221002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b376b-1daa-4ca0-bd83-16a87a780876 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:23.222160 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b376b-1daa-4ca0-bd83-16a87a780876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:23.223620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b376b-1daa-4ca0-bd83-16a87a780876 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:23.225229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 576/1133] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:41:23 2022] GET /identity/v3/domains/default => generated 200 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:23.231152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462bbe1-e95b-4ed5-be89-42c7ab953a8e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:23.328931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462bbe1-e95b-4ed5-be89-42c7ab953a8e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:23.357779 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:23.362632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fff86c-9088-4497-b494-bfb372134f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:23.363399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fff86c-9088-4497-b494-bfb372134f1a None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:23.363591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fff86c-9088-4497-b494-bfb372134f1a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:23.387370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fff86c-9088-4497-b494-bfb372134f1a None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:23.387370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fff86c-9088-4497-b494-bfb372134f1a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:23.387370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fff86c-9088-4497-b494-bfb372134f1a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:23.455599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 558/1134] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:41:23 2022] POST /identity/v3/projects => generated 312 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:41:25.726503 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430c8f2-c5a9-411e-8421-7b80cff3ac67 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:25.727098 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430c8f2-c5a9-411e-8421-7b80cff3ac67 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:25.727098 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430c8f2-c5a9-411e-8421-7b80cff3ac67 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:25.728436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 577/1135] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:25.736763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc074a9-179f-4d6e-81c9-9e48ff32b3f4 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:25.737205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc074a9-179f-4d6e-81c9-9e48ff32b3f4 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:25.737298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc074a9-179f-4d6e-81c9-9e48ff32b3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:25.754391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-0dc074a9-179f-4d6e-81c9-9e48ff32b3f4 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:25.755596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0dc074a9-179f-4d6e-81c9-9e48ff32b3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:25.821361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 559/1136] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:25 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:25.846406 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430c8f2-c5a9-411e-8421-7b80cff3ac67 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:25.859308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:25.951364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430c8f2-c5a9-411e-8421-7b80cff3ac67 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:25.986014 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fad375-0f00-4f52-851f-a8dd66273f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:25.986721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad375-0f00-4f52-851f-a8dd66273f2d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:25.987372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad375-0f00-4f52-851f-a8dd66273f2d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:25.987372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad375-0f00-4f52-851f-a8dd66273f2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:25.988547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fad375-0f00-4f52-851f-a8dd66273f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:25.989973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fad375-0f00-4f52-851f-a8dd66273f2d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:25.991607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 578/1137] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:41:25 2022] GET /identity/v3/domains/default => generated 200 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:25.997727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc074a9-179f-4d6e-81c9-9e48ff32b3f4 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:26.099086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc074a9-179f-4d6e-81c9-9e48ff32b3f4 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:26.113507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:26.133753 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f499fa-ef4c-468d-8fb0-90610943ff14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:26.134429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f499fa-ef4c-468d-8fb0-90610943ff14 None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:26.134710 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f499fa-ef4c-468d-8fb0-90610943ff14 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:26.135014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f499fa-ef4c-468d-8fb0-90610943ff14 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:26.135719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f499fa-ef4c-468d-8fb0-90610943ff14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:26.137143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f499fa-ef4c-468d-8fb0-90610943ff14 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:26.265015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 560/1138] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:41:25 2022] POST /identity/v3/users => generated 287 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:41:28.552645 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfad301-5b30-443f-9069-2882dd553e23 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:28.553293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfad301-5b30-443f-9069-2882dd553e23 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:28.553293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfad301-5b30-443f-9069-2882dd553e23 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:28.554536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 579/1139] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:28.559714 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0934e87-d57a-46d3-a12c-d9f5a77d7e55 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:28.560709 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0934e87-d57a-46d3-a12c-d9f5a77d7e55 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:28.560955 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0934e87-d57a-46d3-a12c-d9f5a77d7e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:28.578157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a0934e87-d57a-46d3-a12c-d9f5a77d7e55 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:28.579389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a0934e87-d57a-46d3-a12c-d9f5a77d7e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:28.646149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 561/1140] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:28 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:28.670064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfad301-5b30-443f-9069-2882dd553e23 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:28.755723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:28.778239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfad301-5b30-443f-9069-2882dd553e23 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:28.818982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f702fce-13b8-4419-8c96-4d0723eeab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:28.819693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f702fce-13b8-4419-8c96-4d0723eeab6b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:28.819972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f702fce-13b8-4419-8c96-4d0723eeab6b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:28.820242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f702fce-13b8-4419-8c96-4d0723eeab6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:28.830440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f702fce-13b8-4419-8c96-4d0723eeab6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-8f702fce-13b8-4419-8c96-4d0723eeab6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:41:28.837931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:28.839147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:28.840225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 580/1141] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:41:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:41:28.845740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0934e87-d57a-46d3-a12c-d9f5a77d7e55 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:28.944693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:28.947600 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0934e87-d57a-46d3-a12c-d9f5a77d7e55 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:28.981330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33380c-d494-4273-90fc-cc426a78f2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:28.982627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33380c-d494-4273-90fc-cc426a78f2fe None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:28.983115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33380c-d494-4273-90fc-cc426a78f2fe None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:28.983391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33380c-d494-4273-90fc-cc426a78f2fe None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:28.984134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33380c-d494-4273-90fc-cc426a78f2fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:28.984436 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33380c-d494-4273-90fc-cc426a78f2fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:28.985792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33380c-d494-4273-90fc-cc426a78f2fe None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:28.994015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 562/1142] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:41:28 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:29.001026 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f702fce-13b8-4419-8c96-4d0723eeab6b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:29.102322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f702fce-13b8-4419-8c96-4d0723eeab6b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:29.136496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0dd79-3cfa-4a93-bd4e-3978c2e09253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:29.137362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0dd79-3cfa-4a93-bd4e-3978c2e09253 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:29.137636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0dd79-3cfa-4a93-bd4e-3978c2e09253 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:29.137904 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0dd79-3cfa-4a93-bd4e-3978c2e09253 None admin] PATH_INFO: `/v3/users/40b54d9b074149dc8494d19fbce5be66` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:29.145716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:29.151312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0dd79-3cfa-4a93-bd4e-3978c2e09253 None admin] RBAC: Authorizing `identity:get_user(user_id=40b54d9b074149dc8494d19fbce5be66)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:29.152963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0dd79-3cfa-4a93-bd4e-3978c2e09253 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:29.155526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 581/1143] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:28 2022] GET /identity/v3/users/40b54d9b074149dc8494d19fbce5be66 => generated 287 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:29.161477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33380c-d494-4273-90fc-cc426a78f2fe None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:29.265650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33380c-d494-4273-90fc-cc426a78f2fe None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:29.301013 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedc7b5-09c2-4ae0-bf51-dc325469f958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:29.301663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedc7b5-09c2-4ae0-bf51-dc325469f958 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:29.301939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedc7b5-09c2-4ae0-bf51-dc325469f958 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:29.302212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedc7b5-09c2-4ae0-bf51-dc325469f958 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:29.334135 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:29.373904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedc7b5-09c2-4ae0-bf51-dc325469f958 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:29.375410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedc7b5-09c2-4ae0-bf51-dc325469f958 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:29.474885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 563/1144] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:41:29 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:41:29.480759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0dd79-3cfa-4a93-bd4e-3978c2e09253 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:29.590699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0dd79-3cfa-4a93-bd4e-3978c2e09253 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:29.626015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76fda6-e3f1-4a61-9887-a733ec1308da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:29.626805 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76fda6-e3f1-4a61-9887-a733ec1308da None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:29.627113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76fda6-e3f1-4a61-9887-a733ec1308da None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:29.627384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76fda6-e3f1-4a61-9887-a733ec1308da None admin] PATH_INFO: `/v3/projects/325f6b6f61b84e8cb973587451b9f411` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:29.628556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76fda6-e3f1-4a61-9887-a733ec1308da None admin] RBAC: Authorizing `identity:get_project(project_id=325f6b6f61b84e8cb973587451b9f411)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:29.629993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76fda6-e3f1-4a61-9887-a733ec1308da None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:29.631725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 582/1145] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:41:29 2022] GET /identity/v3/projects/325f6b6f61b84e8cb973587451b9f411 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:29.637899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedc7b5-09c2-4ae0-bf51-dc325469f958 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:29.701726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:29.740321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedc7b5-09c2-4ae0-bf51-dc325469f958 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:29.773656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2871b9-5127-4d39-b4e6-9b7cc2bcf963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:29.774315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2871b9-5127-4d39-b4e6-9b7cc2bcf963 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:29.774595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2871b9-5127-4d39-b4e6-9b7cc2bcf963 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:29.774893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2871b9-5127-4d39-b4e6-9b7cc2bcf963 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:29.775714 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2871b9-5127-4d39-b4e6-9b7cc2bcf963 None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=325f6b6f61b84e8cb973587451b9f411, user.id=40b54d9b074149dc8494d19fbce5be66) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:29.776025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2871b9-5127-4d39-b4e6-9b7cc2bcf963 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:29.777440 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2871b9-5127-4d39-b4e6-9b7cc2bcf963 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:29.783656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 564/1146] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:41:29 2022] GET /identity/v3/role_assignments?user.id=40b54d9b074149dc8494d19fbce5be66&scope.project.id=325f6b6f61b84e8cb973587451b9f411&role.id=4f574961798246a4b12ad2276e67685d => generated 260 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:31.979784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cf13d-8a57-4d9a-9b7c-085feaf185f2 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:31.980365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cf13d-8a57-4d9a-9b7c-085feaf185f2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:31.980365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cf13d-8a57-4d9a-9b7c-085feaf185f2 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:31.982206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 583/1147] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:31.987998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c84482-08f9-4b0d-8957-56577ad4a432 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:31.988472 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c84482-08f9-4b0d-8957-56577ad4a432 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:31.988588 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c84482-08f9-4b0d-8957-56577ad4a432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:32.005999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-19c84482-08f9-4b0d-8957-56577ad4a432 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:32.007221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-19c84482-08f9-4b0d-8957-56577ad4a432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:32.075435 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 565/1148] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:31 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:32.098440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cf13d-8a57-4d9a-9b7c-085feaf185f2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:32.164350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:32.207643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cf13d-8a57-4d9a-9b7c-085feaf185f2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:32.242646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5bce6-f7ff-4614-8454-646f16608e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:32.243389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5bce6-f7ff-4614-8454-646f16608e7a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:32.243669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5bce6-f7ff-4614-8454-646f16608e7a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:32.243941 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5bce6-f7ff-4614-8454-646f16608e7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:32.254498 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5bce6-f7ff-4614-8454-646f16608e7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-48d5bce6-f7ff-4614-8454-646f16608e7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:41:32.262034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:32.263477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:32.264553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 584/1149] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:41:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:32.270067 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c84482-08f9-4b0d-8957-56577ad4a432 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:32.326930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:32.372619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c84482-08f9-4b0d-8957-56577ad4a432 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:32.406253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa47dd-f97a-4a80-a2e1-76b85e4a3e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:32.406942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa47dd-f97a-4a80-a2e1-76b85e4a3e1a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:32.407225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa47dd-f97a-4a80-a2e1-76b85e4a3e1a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:32.407499 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa47dd-f97a-4a80-a2e1-76b85e4a3e1a None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:32.408242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa47dd-f97a-4a80-a2e1-76b85e4a3e1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:32.408547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa47dd-f97a-4a80-a2e1-76b85e4a3e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:32.409933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa47dd-f97a-4a80-a2e1-76b85e4a3e1a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:32.420189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 566/1150] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:41:32 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:32.426454 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5bce6-f7ff-4614-8454-646f16608e7a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:32.527393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5bce6-f7ff-4614-8454-646f16608e7a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:32.550744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:32.561243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4abdc1-3393-4568-ad7a-68c329644f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:32.572811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4abdc1-3393-4568-ad7a-68c329644f09 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:32.572811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4abdc1-3393-4568-ad7a-68c329644f09 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:32.572811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4abdc1-3393-4568-ad7a-68c329644f09 None admin] PATH_INFO: `/v3/users/40b54d9b074149dc8494d19fbce5be66` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:32.572811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4abdc1-3393-4568-ad7a-68c329644f09 None admin] RBAC: Authorizing `identity:get_user(user_id=40b54d9b074149dc8494d19fbce5be66)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:32.572811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4abdc1-3393-4568-ad7a-68c329644f09 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:32.572811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 585/1151] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:32 2022] GET /identity/v3/users/40b54d9b074149dc8494d19fbce5be66 => generated 287 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:32.573327 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa47dd-f97a-4a80-a2e1-76b85e4a3e1a None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:32.675871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa47dd-f97a-4a80-a2e1-76b85e4a3e1a None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:32.694668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:32.712685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b23c2e-92ad-45e4-8e0a-8e26dec3dfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:32.713554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b23c2e-92ad-45e4-8e0a-8e26dec3dfe9 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:32.713839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b23c2e-92ad-45e4-8e0a-8e26dec3dfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:32.714119 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b23c2e-92ad-45e4-8e0a-8e26dec3dfe9 None admin] PATH_INFO: `/v3/projects/325f6b6f61b84e8cb973587451b9f411` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:32.715315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b23c2e-92ad-45e4-8e0a-8e26dec3dfe9 None admin] RBAC: Authorizing `identity:get_project(project_id=325f6b6f61b84e8cb973587451b9f411)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:32.717799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b23c2e-92ad-45e4-8e0a-8e26dec3dfe9 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:32.719548 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 567/1152] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:41:32 2022] GET /identity/v3/projects/325f6b6f61b84e8cb973587451b9f411 => generated 312 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:32.725481 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4abdc1-3393-4568-ad7a-68c329644f09 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:32.832532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4abdc1-3393-4568-ad7a-68c329644f09 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:32.867299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ba458-95ef-4e41-b52a-13ac68faf07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:32.868197 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ba458-95ef-4e41-b52a-13ac68faf07e None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:32.868478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ba458-95ef-4e41-b52a-13ac68faf07e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:32.868756 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ba458-95ef-4e41-b52a-13ac68faf07e None admin] PATH_INFO: `/v3/projects/325f6b6f61b84e8cb973587451b9f411/users/40b54d9b074149dc8494d19fbce5be66/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:32.870723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ba458-95ef-4e41-b52a-13ac68faf07e None admin] RBAC: Authorizing `identity:create_grant(project_id=325f6b6f61b84e8cb973587451b9f411, user_id=40b54d9b074149dc8494d19fbce5be66, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:32.872729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ba458-95ef-4e41-b52a-13ac68faf07e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:32.975475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 586/1153] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:41:32 2022] PUT /identity/v3/projects/325f6b6f61b84e8cb973587451b9f411/users/40b54d9b074149dc8494d19fbce5be66/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:41:35.151538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72116002-0994-4af1-b5d1-a5c4b1121726 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:35.152090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72116002-0994-4af1-b5d1-a5c4b1121726 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:35.152090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72116002-0994-4af1-b5d1-a5c4b1121726 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:35.153990 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 568/1154] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:35 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:35.159303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca05bc-b99a-4008-b7b1-a462b7c81ecf None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:35.159724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca05bc-b99a-4008-b7b1-a462b7c81ecf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:35.159856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca05bc-b99a-4008-b7b1-a462b7c81ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:35.177422 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-36ca05bc-b99a-4008-b7b1-a462b7c81ecf None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:35.178631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-36ca05bc-b99a-4008-b7b1-a462b7c81ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:35.225443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:35.244963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 587/1155] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:35 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:35.269498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72116002-0994-4af1-b5d1-a5c4b1121726 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:35.371910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72116002-0994-4af1-b5d1-a5c4b1121726 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:35.406169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a8dc5-cb09-4fa5-b689-32a3a1fef27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:35.406994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a8dc5-cb09-4fa5-b689-32a3a1fef27b None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:35.407273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a8dc5-cb09-4fa5-b689-32a3a1fef27b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:35.407542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a8dc5-cb09-4fa5-b689-32a3a1fef27b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:35.417688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a8dc5-cb09-4fa5-b689-32a3a1fef27b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-874a8dc5-cb09-4fa5-b689-32a3a1fef27b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:41:35.425869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:35.427034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:41:35.428085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 569/1156] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:41:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:35.433112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca05bc-b99a-4008-b7b1-a462b7c81ecf None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:35.531555 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca05bc-b99a-4008-b7b1-a462b7c81ecf None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:35.563948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42c959-9871-4f1f-8632-3ea005aeef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:35.564596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42c959-9871-4f1f-8632-3ea005aeef0b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:35.564874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42c959-9871-4f1f-8632-3ea005aeef0b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:35.565144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42c959-9871-4f1f-8632-3ea005aeef0b None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:35.565877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42c959-9871-4f1f-8632-3ea005aeef0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:35.566176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42c959-9871-4f1f-8632-3ea005aeef0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:35.567564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42c959-9871-4f1f-8632-3ea005aeef0b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:35.576250 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 588/1157] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:41:35 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:35.582054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a8dc5-cb09-4fa5-b689-32a3a1fef27b None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:35.628860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:35.682320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a8dc5-cb09-4fa5-b689-32a3a1fef27b None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:35.716003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b78f2c-82b5-4f0e-98b6-2198f0f48055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:35.717040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b78f2c-82b5-4f0e-98b6-2198f0f48055 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:35.717320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b78f2c-82b5-4f0e-98b6-2198f0f48055 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:35.717595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b78f2c-82b5-4f0e-98b6-2198f0f48055 None admin] PATH_INFO: `/v3/users/40b54d9b074149dc8494d19fbce5be66` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:35.718765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b78f2c-82b5-4f0e-98b6-2198f0f48055 None admin] RBAC: Authorizing `identity:get_user(user_id=40b54d9b074149dc8494d19fbce5be66)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:35.720364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b78f2c-82b5-4f0e-98b6-2198f0f48055 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:35.721984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 570/1158] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:35 2022] GET /identity/v3/users/40b54d9b074149dc8494d19fbce5be66 => generated 287 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:35.727739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42c959-9871-4f1f-8632-3ea005aeef0b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:35.773185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:35.828312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42c959-9871-4f1f-8632-3ea005aeef0b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:35.862156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fedaa6-8e5f-4e77-9ea6-68d10f5926ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:35.862788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fedaa6-8e5f-4e77-9ea6-68d10f5926ed None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:35.863095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fedaa6-8e5f-4e77-9ea6-68d10f5926ed None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:35.863363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fedaa6-8e5f-4e77-9ea6-68d10f5926ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:35.930705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fedaa6-8e5f-4e77-9ea6-68d10f5926ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:35.932080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fedaa6-8e5f-4e77-9ea6-68d10f5926ed None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:35.976860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:36.031450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 589/1159] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:41:35 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:41:36.037543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b78f2c-82b5-4f0e-98b6-2198f0f48055 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:36.140382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b78f2c-82b5-4f0e-98b6-2198f0f48055 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:36.154221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:36.175035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464e5ae-9698-409e-b0c2-7c3adcc93c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:36.175907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464e5ae-9698-409e-b0c2-7c3adcc93c8f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:36.176183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464e5ae-9698-409e-b0c2-7c3adcc93c8f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:36.176453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464e5ae-9698-409e-b0c2-7c3adcc93c8f None admin] PATH_INFO: `/v3/projects/325f6b6f61b84e8cb973587451b9f411` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:36.177639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464e5ae-9698-409e-b0c2-7c3adcc93c8f None admin] RBAC: Authorizing `identity:get_project(project_id=325f6b6f61b84e8cb973587451b9f411)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:36.179096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464e5ae-9698-409e-b0c2-7c3adcc93c8f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:36.180855 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 571/1160] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:41:36 2022] GET /identity/v3/projects/325f6b6f61b84e8cb973587451b9f411 => generated 312 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:36.187022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fedaa6-8e5f-4e77-9ea6-68d10f5926ed None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:36.284147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fedaa6-8e5f-4e77-9ea6-68d10f5926ed None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:36.318327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739c043-4fed-4573-a891-a545596b05d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:36.319016 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739c043-4fed-4573-a891-a545596b05d8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:36.319293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739c043-4fed-4573-a891-a545596b05d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:36.319558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739c043-4fed-4573-a891-a545596b05d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:36.320378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739c043-4fed-4573-a891-a545596b05d8 None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=325f6b6f61b84e8cb973587451b9f411, user.id=40b54d9b074149dc8494d19fbce5be66) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:36.320684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739c043-4fed-4573-a891-a545596b05d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:36.322096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739c043-4fed-4573-a891-a545596b05d8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:36.328756 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 590/1161] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:41:36 2022] GET /identity/v3/role_assignments?user.id=40b54d9b074149dc8494d19fbce5be66&scope.project.id=325f6b6f61b84e8cb973587451b9f411&role.id=4f574961798246a4b12ad2276e67685d => generated 610 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:38.539847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de712389-fe6d-4a4a-9cd4-c3d95aada2fa None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:38.540405 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de712389-fe6d-4a4a-9cd4-c3d95aada2fa None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:38.540405 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de712389-fe6d-4a4a-9cd4-c3d95aada2fa None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:38.541779 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 572/1162] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:38.547727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf1d1a-2047-4818-a31a-134cd21ddf9a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:38.548147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf1d1a-2047-4818-a31a-134cd21ddf9a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:38.548277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf1d1a-2047-4818-a31a-134cd21ddf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:38.565377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-92cf1d1a-2047-4818-a31a-134cd21ddf9a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:38.566559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-92cf1d1a-2047-4818-a31a-134cd21ddf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:38.568737 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:38.633362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 591/1163] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:38 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:38.656185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de712389-fe6d-4a4a-9cd4-c3d95aada2fa None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:38.758356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de712389-fe6d-4a4a-9cd4-c3d95aada2fa None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:38.791797 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea22c9-cc62-4296-aeeb-19f1f5ddf86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:38.792366 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea22c9-cc62-4296-aeeb-19f1f5ddf86e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:38.792640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea22c9-cc62-4296-aeeb-19f1f5ddf86e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:38.792907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea22c9-cc62-4296-aeeb-19f1f5ddf86e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:38.842381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:38.861053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea22c9-cc62-4296-aeeb-19f1f5ddf86e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:38.862552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea22c9-cc62-4296-aeeb-19f1f5ddf86e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:38.959227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 573/1164] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:41:38 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:41:38.965960 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf1d1a-2047-4818-a31a-134cd21ddf9a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:39.066241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf1d1a-2047-4818-a31a-134cd21ddf9a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:39.100410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e7c7c-0bfc-49c4-801e-c82dc79cf8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:39.101211 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e7c7c-0bfc-49c4-801e-c82dc79cf8f4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:39.101494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e7c7c-0bfc-49c4-801e-c82dc79cf8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:39.101768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e7c7c-0bfc-49c4-801e-c82dc79cf8f4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:39.111572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e7c7c-0bfc-49c4-801e-c82dc79cf8f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:39.111572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e7c7c-0bfc-49c4-801e-c82dc79cf8f4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:39.112630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f19e7c7c-0bfc-49c4-801e-c82dc79cf8f4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:39.126028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:39.127308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:39.128518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 592/1165] 10.222.0.121 () {56 vars in 1122 bytes} [Thu May 26 17:41:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:39.133273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea22c9-cc62-4296-aeeb-19f1f5ddf86e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:39.220285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:39.237846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea22c9-cc62-4296-aeeb-19f1f5ddf86e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:39.271011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0442e-21bc-4c04-952a-e96a0ab855a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:39.271666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0442e-21bc-4c04-952a-e96a0ab855a5 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:39.271943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0442e-21bc-4c04-952a-e96a0ab855a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:39.272214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0442e-21bc-4c04-952a-e96a0ab855a5 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:39.272916 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0442e-21bc-4c04-952a-e96a0ab855a5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:39.273215 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0442e-21bc-4c04-952a-e96a0ab855a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:39.274560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0442e-21bc-4c04-952a-e96a0ab855a5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:39.282151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 574/1166] 10.222.0.121 () {56 vars in 1120 bytes} [Thu May 26 17:41:39 2022] GET /identity/v3/domains?name=swift_test => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:41.488374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5414b-74ca-4f6d-9df9-bb4118e1a82b None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:41.488931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5414b-74ca-4f6d-9df9-bb4118e1a82b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:41.488931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5414b-74ca-4f6d-9df9-bb4118e1a82b None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:41.490276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 593/1167] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:41.495967 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12effcef-a237-449d-b351-6c11cd96512a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:41.496365 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12effcef-a237-449d-b351-6c11cd96512a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:41.496528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12effcef-a237-449d-b351-6c11cd96512a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:41.513695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-12effcef-a237-449d-b351-6c11cd96512a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:41.514899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-12effcef-a237-449d-b351-6c11cd96512a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:41.581197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 575/1168] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:41 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:41.607070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5414b-74ca-4f6d-9df9-bb4118e1a82b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:41.706128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5414b-74ca-4f6d-9df9-bb4118e1a82b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:41.739217 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a56e80-6161-48a6-a667-abda44886f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:41.739803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a56e80-6161-48a6-a667-abda44886f02 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:41.740075 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a56e80-6161-48a6-a667-abda44886f02 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:41.740345 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a56e80-6161-48a6-a667-abda44886f02 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:41.741082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a56e80-6161-48a6-a667-abda44886f02 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:41.742439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a56e80-6161-48a6-a667-abda44886f02 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:41.870618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 594/1169] 10.222.0.121 () {60 vars in 1143 bytes} [Thu May 26 17:41:41 2022] POST /identity/v3/domains => generated 268 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:41:44.097700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e6bbe-0f7b-4ede-828c-8274c137e09a None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:44.098246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e6bbe-0f7b-4ede-828c-8274c137e09a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:44.098246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e6bbe-0f7b-4ede-828c-8274c137e09a None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:44.099636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 576/1170] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:44.104991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15152a65-9bf5-4215-b406-e1a53fec8fa2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:44.105417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15152a65-9bf5-4215-b406-e1a53fec8fa2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:44.105551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15152a65-9bf5-4215-b406-e1a53fec8fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:44.123631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-15152a65-9bf5-4215-b406-e1a53fec8fa2 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:44.124827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-15152a65-9bf5-4215-b406-e1a53fec8fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:44.179300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:44.192655 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 595/1171] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:44 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:44.216626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e6bbe-0f7b-4ede-828c-8274c137e09a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:44.295228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:44.318028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e6bbe-0f7b-4ede-828c-8274c137e09a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:44.352106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981de1b-b766-4649-858a-44e51e1b3ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:44.352880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981de1b-b766-4649-858a-44e51e1b3ffc None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:44.353161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981de1b-b766-4649-858a-44e51e1b3ffc None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:44.353433 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981de1b-b766-4649-858a-44e51e1b3ffc None admin] PATH_INFO: `/v3/domains/95ff06e3bf2d48cc8fa956e56f122d86` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:44.364583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981de1b-b766-4649-858a-44e51e1b3ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=95ff06e3bf2d48cc8fa956e56f122d86)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:44.366019 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981de1b-b766-4649-858a-44e51e1b3ffc None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:44.367704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 577/1172] 10.222.0.121 () {56 vars in 1188 bytes} [Thu May 26 17:41:44 2022] GET /identity/v3/domains/95ff06e3bf2d48cc8fa956e56f122d86 => generated 268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:44.373848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15152a65-9bf5-4215-b406-e1a53fec8fa2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:44.472077 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15152a65-9bf5-4215-b406-e1a53fec8fa2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:44.505886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50ef3b-15eb-4cdc-a7e7-756860fad88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:44.506622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50ef3b-15eb-4cdc-a7e7-756860fad88e None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:44.506889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50ef3b-15eb-4cdc-a7e7-756860fad88e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:44.507167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50ef3b-15eb-4cdc-a7e7-756860fad88e None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:44.507878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50ef3b-15eb-4cdc-a7e7-756860fad88e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:44.509319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50ef3b-15eb-4cdc-a7e7-756860fad88e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:44.607276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 596/1173] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:41:44 2022] POST /identity/v3/projects => generated 362 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:41:46.862550 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ee753-4913-40e6-b8d0-559585d169b0 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:46.863118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ee753-4913-40e6-b8d0-559585d169b0 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:46.863118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ee753-4913-40e6-b8d0-559585d169b0 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:46.864481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 578/1174] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:46.869698 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f266def-31a1-4ead-b1c1-9f2cf2efb4c7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:46.870107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f266def-31a1-4ead-b1c1-9f2cf2efb4c7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:46.870258 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f266def-31a1-4ead-b1c1-9f2cf2efb4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:46.888227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3f266def-31a1-4ead-b1c1-9f2cf2efb4c7 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:46.889421 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3f266def-31a1-4ead-b1c1-9f2cf2efb4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:46.924960 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:46.958948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 597/1175] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:46 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:46.984912 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ee753-4913-40e6-b8d0-559585d169b0 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:47.085504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ee753-4913-40e6-b8d0-559585d169b0 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:47.120234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664be05-8279-4033-9a92-380f14d17164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:47.121096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664be05-8279-4033-9a92-380f14d17164 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:47.124195 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664be05-8279-4033-9a92-380f14d17164 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:47.124195 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664be05-8279-4033-9a92-380f14d17164 None admin] PATH_INFO: `/v3/domains/95ff06e3bf2d48cc8fa956e56f122d86` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:47.124195 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664be05-8279-4033-9a92-380f14d17164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=95ff06e3bf2d48cc8fa956e56f122d86)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:47.125200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664be05-8279-4033-9a92-380f14d17164 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:47.126832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 579/1176] 10.222.0.121 () {56 vars in 1188 bytes} [Thu May 26 17:41:46 2022] GET /identity/v3/domains/95ff06e3bf2d48cc8fa956e56f122d86 => generated 268 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:41:47.132537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f266def-31a1-4ead-b1c1-9f2cf2efb4c7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:47.229507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f266def-31a1-4ead-b1c1-9f2cf2efb4c7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:47.262409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8c4fb-5f72-495f-9cb4-289275ec798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:47.263102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8c4fb-5f72-495f-9cb4-289275ec798a None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:47.263384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8c4fb-5f72-495f-9cb4-289275ec798a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:47.263655 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8c4fb-5f72-495f-9cb4-289275ec798a None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:47.264365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8c4fb-5f72-495f-9cb4-289275ec798a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:47.265809 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8c4fb-5f72-495f-9cb4-289275ec798a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:47.395007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 598/1177] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:41:47 2022] POST /identity/v3/users => generated 312 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:41:49.654727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e793d-b7e9-4345-93cb-d3a4f4af9526 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:49.655306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e793d-b7e9-4345-93cb-d3a4f4af9526 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:49.655306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e793d-b7e9-4345-93cb-d3a4f4af9526 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:49.656692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 580/1178] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:49.661950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658978e5-df41-4839-a2e8-8693a4d5c6ba None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:49.662379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658978e5-df41-4839-a2e8-8693a4d5c6ba None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:49.662508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658978e5-df41-4839-a2e8-8693a4d5c6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:49.680208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-658978e5-df41-4839-a2e8-8693a4d5c6ba None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:49.681397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-658978e5-df41-4839-a2e8-8693a4d5c6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:49.702610 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:49.748340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 599/1179] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:49 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:49.771665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e793d-b7e9-4345-93cb-d3a4f4af9526 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:49.775588 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:49.873156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e793d-b7e9-4345-93cb-d3a4f4af9526 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:49.905549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e478c7-dd6d-48d4-add3-e09c4620a18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:49.906257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e478c7-dd6d-48d4-add3-e09c4620a18e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:49.906532 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e478c7-dd6d-48d4-add3-e09c4620a18e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:49.906802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e478c7-dd6d-48d4-add3-e09c4620a18e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:49.916893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e478c7-dd6d-48d4-add3-e09c4620a18e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-52e478c7-dd6d-48d4-add3-e09c4620a18e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:41:49.925093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:49.926222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:41:49.927544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 581/1180] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:41:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:49.932830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658978e5-df41-4839-a2e8-8693a4d5c6ba None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:50.036501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658978e5-df41-4839-a2e8-8693a4d5c6ba None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:50.070989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1aacd-74b7-479a-be31-6ba74c81fafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:50.071665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1aacd-74b7-479a-be31-6ba74c81fafb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:50.071949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1aacd-74b7-479a-be31-6ba74c81fafb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:50.072228 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1aacd-74b7-479a-be31-6ba74c81fafb None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:50.072973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1aacd-74b7-479a-be31-6ba74c81fafb None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:50.073274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1aacd-74b7-479a-be31-6ba74c81fafb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:50.074690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1aacd-74b7-479a-be31-6ba74c81fafb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:50.086801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 600/1181] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:41:49 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:50.093444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e478c7-dd6d-48d4-add3-e09c4620a18e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:50.163368 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:50.200362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e478c7-dd6d-48d4-add3-e09c4620a18e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:50.234176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9c69b-3b98-427c-b6a2-4b4a74a3efff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:50.235075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c69b-3b98-427c-b6a2-4b4a74a3efff None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:50.235354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c69b-3b98-427c-b6a2-4b4a74a3efff None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:50.235623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c69b-3b98-427c-b6a2-4b4a74a3efff None admin] PATH_INFO: `/v3/users/74ed92f14e194396b40775b190a48e5f` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:50.250592 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9c69b-3b98-427c-b6a2-4b4a74a3efff None admin] RBAC: Authorizing `identity:get_user(user_id=74ed92f14e194396b40775b190a48e5f)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:50.252072 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9c69b-3b98-427c-b6a2-4b4a74a3efff None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:50.253756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 582/1182] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:50 2022] GET /identity/v3/users/74ed92f14e194396b40775b190a48e5f => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:50.259531 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1aacd-74b7-479a-be31-6ba74c81fafb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:50.279900 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:50.360535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1aacd-74b7-479a-be31-6ba74c81fafb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:50.393703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c651826-1fd2-4722-b707-97d240e30629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:50.394349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c651826-1fd2-4722-b707-97d240e30629 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:50.394637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c651826-1fd2-4722-b707-97d240e30629 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:50.394942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c651826-1fd2-4722-b707-97d240e30629 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:50.461421 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c651826-1fd2-4722-b707-97d240e30629 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:50.462807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c651826-1fd2-4722-b707-97d240e30629 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:50.481587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:50.560595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 601/1183] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:41:50 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:41:50.566422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9c69b-3b98-427c-b6a2-4b4a74a3efff None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:50.663382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9c69b-3b98-427c-b6a2-4b4a74a3efff None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:50.686144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:50.697164 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8be4bd-992f-48ff-822a-fd0a0b00c517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:50.699555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8be4bd-992f-48ff-822a-fd0a0b00c517 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:50.699555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8be4bd-992f-48ff-822a-fd0a0b00c517 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:50.699555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8be4bd-992f-48ff-822a-fd0a0b00c517 None admin] PATH_INFO: `/v3/projects/ffdb7ec464354ceaab9e07dd7ffbdeae` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:50.700453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8be4bd-992f-48ff-822a-fd0a0b00c517 None admin] RBAC: Authorizing `identity:get_project(project_id=ffdb7ec464354ceaab9e07dd7ffbdeae)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:50.701932 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8be4bd-992f-48ff-822a-fd0a0b00c517 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:50.703667 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 583/1184] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:41:50 2022] GET /identity/v3/projects/ffdb7ec464354ceaab9e07dd7ffbdeae => generated 362 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:50.709762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c651826-1fd2-4722-b707-97d240e30629 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:50.807400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c651826-1fd2-4722-b707-97d240e30629 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:50.824959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:50.841917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0419b2-53ad-4b72-b4d5-4d196484b4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:50.842583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0419b2-53ad-4b72-b4d5-4d196484b4ce None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:50.842898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0419b2-53ad-4b72-b4d5-4d196484b4ce None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:50.859565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0419b2-53ad-4b72-b4d5-4d196484b4ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:50.859565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0419b2-53ad-4b72-b4d5-4d196484b4ce None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=ffdb7ec464354ceaab9e07dd7ffbdeae, user.id=74ed92f14e194396b40775b190a48e5f) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:50.859565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0419b2-53ad-4b72-b4d5-4d196484b4ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:50.859565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0419b2-53ad-4b72-b4d5-4d196484b4ce None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:50.859565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 602/1185] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:41:50 2022] GET /identity/v3/role_assignments?user.id=74ed92f14e194396b40775b190a48e5f&scope.project.id=ffdb7ec464354ceaab9e07dd7ffbdeae&role.id=4f574961798246a4b12ad2276e67685d => generated 260 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:53.048586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1e1e2-35f1-493c-8e83-532d2d6e07e8 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:53.049151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1e1e2-35f1-493c-8e83-532d2d6e07e8 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:53.049151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1e1e2-35f1-493c-8e83-532d2d6e07e8 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:53.050508 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 584/1186] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:53.056038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e788aa3-595a-4ab3-abda-c754c8ecc5e7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:53.056473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e788aa3-595a-4ab3-abda-c754c8ecc5e7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:53.056567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e788aa3-595a-4ab3-abda-c754c8ecc5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:53.073839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2e788aa3-595a-4ab3-abda-c754c8ecc5e7 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:53.075098 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2e788aa3-595a-4ab3-abda-c754c8ecc5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:53.141983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 603/1187] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:53 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:53.166670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1e1e2-35f1-493c-8e83-532d2d6e07e8 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:53.267813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1e1e2-35f1-493c-8e83-532d2d6e07e8 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:53.302661 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28252140-b972-4dea-a260-da5c2dc907e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:53.303396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28252140-b972-4dea-a260-da5c2dc907e0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:53.303677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28252140-b972-4dea-a260-da5c2dc907e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:53.303951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28252140-b972-4dea-a260-da5c2dc907e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:53.314363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28252140-b972-4dea-a260-da5c2dc907e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-28252140-b972-4dea-a260-da5c2dc907e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:41:53.321863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:53.323094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:41:53.324205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 585/1188] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:41:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:53.329298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e788aa3-595a-4ab3-abda-c754c8ecc5e7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:53.427316 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e788aa3-595a-4ab3-abda-c754c8ecc5e7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:53.440103 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:53.460768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f43385-8c58-43e3-99cd-5c2921bc8aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:53.461559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f43385-8c58-43e3-99cd-5c2921bc8aed None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:53.461838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f43385-8c58-43e3-99cd-5c2921bc8aed None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:53.462137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f43385-8c58-43e3-99cd-5c2921bc8aed None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:53.462960 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f43385-8c58-43e3-99cd-5c2921bc8aed None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:53.463271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f43385-8c58-43e3-99cd-5c2921bc8aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:53.465304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f43385-8c58-43e3-99cd-5c2921bc8aed None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:53.473741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 604/1189] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:41:53 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:53.479757 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28252140-b972-4dea-a260-da5c2dc907e0 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:53.532023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:53.582042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28252140-b972-4dea-a260-da5c2dc907e0 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:53.616759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dff373-3b39-4e72-84b5-9c0d23b10a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:53.617629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dff373-3b39-4e72-84b5-9c0d23b10a2c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:53.617913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dff373-3b39-4e72-84b5-9c0d23b10a2c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:53.618191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dff373-3b39-4e72-84b5-9c0d23b10a2c None admin] PATH_INFO: `/v3/users/74ed92f14e194396b40775b190a48e5f` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:53.619387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dff373-3b39-4e72-84b5-9c0d23b10a2c None admin] RBAC: Authorizing `identity:get_user(user_id=74ed92f14e194396b40775b190a48e5f)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:53.620853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dff373-3b39-4e72-84b5-9c0d23b10a2c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:53.622525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 586/1190] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:53 2022] GET /identity/v3/users/74ed92f14e194396b40775b190a48e5f => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:53.628387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f43385-8c58-43e3-99cd-5c2921bc8aed None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:53.727435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f43385-8c58-43e3-99cd-5c2921bc8aed None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:53.760709 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4057de-570f-4f04-81f5-f78dde67c9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:53.761564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4057de-570f-4f04-81f5-f78dde67c9e8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:53.762378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4057de-570f-4f04-81f5-f78dde67c9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:53.762723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4057de-570f-4f04-81f5-f78dde67c9e8 None admin] PATH_INFO: `/v3/projects/ffdb7ec464354ceaab9e07dd7ffbdeae` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:53.764047 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4057de-570f-4f04-81f5-f78dde67c9e8 None admin] RBAC: Authorizing `identity:get_project(project_id=ffdb7ec464354ceaab9e07dd7ffbdeae)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:53.765471 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4057de-570f-4f04-81f5-f78dde67c9e8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:53.767220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 605/1191] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:41:53 2022] GET /identity/v3/projects/ffdb7ec464354ceaab9e07dd7ffbdeae => generated 362 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:53.773612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dff373-3b39-4e72-84b5-9c0d23b10a2c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:53.873558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dff373-3b39-4e72-84b5-9c0d23b10a2c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:53.890394 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:53.906565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3980b43-2732-4b0b-a58f-4308570da0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:53.907478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3980b43-2732-4b0b-a58f-4308570da0a2 None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:53.907755 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3980b43-2732-4b0b-a58f-4308570da0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:53.908027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3980b43-2732-4b0b-a58f-4308570da0a2 None admin] PATH_INFO: `/v3/projects/ffdb7ec464354ceaab9e07dd7ffbdeae/users/74ed92f14e194396b40775b190a48e5f/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:53.909942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3980b43-2732-4b0b-a58f-4308570da0a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ffdb7ec464354ceaab9e07dd7ffbdeae, user_id=74ed92f14e194396b40775b190a48e5f, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:53.911392 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3980b43-2732-4b0b-a58f-4308570da0a2 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:53.988188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 587/1192] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:41:53 2022] PUT /identity/v3/projects/ffdb7ec464354ceaab9e07dd7ffbdeae/users/74ed92f14e194396b40775b190a48e5f/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:41:56.191802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b825214-78f2-4e5d-9d8a-ba39b5200a86 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:56.192402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b825214-78f2-4e5d-9d8a-ba39b5200a86 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:56.192402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b825214-78f2-4e5d-9d8a-ba39b5200a86 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:56.193713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 606/1193] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:41:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:41:56.199412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7fec8-c239-44cc-aa44-c1c31c933ee1 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:56.199885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7fec8-c239-44cc-aa44-c1c31c933ee1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:56.200002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7fec8-c239-44cc-aa44-c1c31c933ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:56.217638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-9ab7fec8-c239-44cc-aa44-c1c31c933ee1 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:41:56.218842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9ab7fec8-c239-44cc-aa44-c1c31c933ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:41:56.288752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 588/1194] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:41:56 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:41:56.313895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b825214-78f2-4e5d-9d8a-ba39b5200a86 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:56.367300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:56.427061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b825214-78f2-4e5d-9d8a-ba39b5200a86 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:56.465826 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ffae0-0ef7-4026-b89d-bf1da050f92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:56.466626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ffae0-0ef7-4026-b89d-bf1da050f92f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:56.466945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ffae0-0ef7-4026-b89d-bf1da050f92f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:56.467204 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ffae0-0ef7-4026-b89d-bf1da050f92f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:56.479165 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ffae0-0ef7-4026-b89d-bf1da050f92f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-985ffae0-0ef7-4026-b89d-bf1da050f92f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:41:56.487232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:41:56.488461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:41:56.489712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 607/1195] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:41:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:41:56.494567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7fec8-c239-44cc-aa44-c1c31c933ee1 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:56.606893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7fec8-c239-44cc-aa44-c1c31c933ee1 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:56.616956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:56.648302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794d49c-bbb9-4d4e-806a-9cc99aa4313d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:56.649008 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794d49c-bbb9-4d4e-806a-9cc99aa4313d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:56.649659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794d49c-bbb9-4d4e-806a-9cc99aa4313d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:56.649979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794d49c-bbb9-4d4e-806a-9cc99aa4313d None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:56.650740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794d49c-bbb9-4d4e-806a-9cc99aa4313d None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:56.651075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794d49c-bbb9-4d4e-806a-9cc99aa4313d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:56.653165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794d49c-bbb9-4d4e-806a-9cc99aa4313d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:56.661859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 589/1196] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:41:56 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:56.668195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ffae0-0ef7-4026-b89d-bf1da050f92f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:56.767372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ffae0-0ef7-4026-b89d-bf1da050f92f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:56.768910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:56.802108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d102e-de4b-430d-8a07-36f6caab1e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:56.803014 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d102e-de4b-430d-8a07-36f6caab1e80 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:56.803440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d102e-de4b-430d-8a07-36f6caab1e80 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:56.803716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d102e-de4b-430d-8a07-36f6caab1e80 None admin] PATH_INFO: `/v3/users/74ed92f14e194396b40775b190a48e5f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:56.804878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d102e-de4b-430d-8a07-36f6caab1e80 None admin] RBAC: Authorizing `identity:get_user(user_id=74ed92f14e194396b40775b190a48e5f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:56.806302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d102e-de4b-430d-8a07-36f6caab1e80 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:56.807935 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 608/1197] 10.222.0.121 () {56 vars in 1182 bytes} [Thu May 26 17:41:56 2022] GET /identity/v3/users/74ed92f14e194396b40775b190a48e5f => generated 312 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:56.814353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794d49c-bbb9-4d4e-806a-9cc99aa4313d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:56.912349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794d49c-bbb9-4d4e-806a-9cc99aa4313d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:56.945369 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845f5ac-9cb9-4ebb-9273-c71399cad866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:56.946090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845f5ac-9cb9-4ebb-9273-c71399cad866 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:56.946302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845f5ac-9cb9-4ebb-9273-c71399cad866 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:56.946572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845f5ac-9cb9-4ebb-9273-c71399cad866 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:56.984653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:57.014575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845f5ac-9cb9-4ebb-9273-c71399cad866 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:57.016039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845f5ac-9cb9-4ebb-9273-c71399cad866 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:57.113779 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 590/1198] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:41:56 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:41:57.119604 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d102e-de4b-430d-8a07-36f6caab1e80 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:57.225534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d102e-de4b-430d-8a07-36f6caab1e80 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:57.259878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b4ce4-f3cb-40bf-bc30-7954ca471fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:57.260742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b4ce4-f3cb-40bf-bc30-7954ca471fba None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:57.261024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b4ce4-f3cb-40bf-bc30-7954ca471fba None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:57.261298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b4ce4-f3cb-40bf-bc30-7954ca471fba None admin] PATH_INFO: `/v3/projects/ffdb7ec464354ceaab9e07dd7ffbdeae` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:57.262581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b4ce4-f3cb-40bf-bc30-7954ca471fba None admin] RBAC: Authorizing `identity:get_project(project_id=ffdb7ec464354ceaab9e07dd7ffbdeae)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:57.264095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b4ce4-f3cb-40bf-bc30-7954ca471fba None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:57.265796 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 609/1199] 10.222.0.121 () {56 vars in 1191 bytes} [Thu May 26 17:41:57 2022] GET /identity/v3/projects/ffdb7ec464354ceaab9e07dd7ffbdeae => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:41:57.271956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845f5ac-9cb9-4ebb-9273-c71399cad866 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:41:57.339391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:41:57.370427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845f5ac-9cb9-4ebb-9273-c71399cad866 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:41:57.403773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93a6cb-7236-4b58-b70b-e0b73399d7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:41:57.404416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93a6cb-7236-4b58-b70b-e0b73399d7b1 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:41:57.404692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93a6cb-7236-4b58-b70b-e0b73399d7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:41:57.404962 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93a6cb-7236-4b58-b70b-e0b73399d7b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:41:57.405775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93a6cb-7236-4b58-b70b-e0b73399d7b1 None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=ffdb7ec464354ceaab9e07dd7ffbdeae, user.id=74ed92f14e194396b40775b190a48e5f) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:41:57.406082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93a6cb-7236-4b58-b70b-e0b73399d7b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:41:57.407575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93a6cb-7236-4b58-b70b-e0b73399d7b1 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:41:57.413371 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 591/1200] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:41:57 2022] GET /identity/v3/role_assignments?user.id=74ed92f14e194396b40775b190a48e5f&scope.project.id=ffdb7ec464354ceaab9e07dd7ffbdeae&role.id=4f574961798246a4b12ad2276e67685d => generated 610 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:42:07.255686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=81 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:42:07.410689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=33 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:17.612327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26357e2f-dc60-4e3e-979a-78f4b63a55c0 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:17.612917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26357e2f-dc60-4e3e-979a-78f4b63a55c0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:17.613027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26357e2f-dc60-4e3e-979a-78f4b63a55c0 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:17.614320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 610/1201] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:43:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:43:17.619663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78106-b343-411a-9ce1-725f24da86de None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:17.620118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78106-b343-411a-9ce1-725f24da86de None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:17.620219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78106-b343-411a-9ce1-725f24da86de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:17.638047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f6c78106-b343-411a-9ce1-725f24da86de None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:43:17.640363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f6c78106-b343-411a-9ce1-725f24da86de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:43:17.705675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 592/1202] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:43:17 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:43:17.730069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26357e2f-dc60-4e3e-979a-78f4b63a55c0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:17.834307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26357e2f-dc60-4e3e-979a-78f4b63a55c0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:17.868269 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763c787-59be-4656-8191-8450ac8d86e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:17.869057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763c787-59be-4656-8191-8450ac8d86e1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:17.869332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763c787-59be-4656-8191-8450ac8d86e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:17.869601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763c787-59be-4656-8191-8450ac8d86e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:17.881008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763c787-59be-4656-8191-8450ac8d86e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:17.882457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763c787-59be-4656-8191-8450ac8d86e1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7763c787-59be-4656-8191-8450ac8d86e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:17.901449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:17.902614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:17.903697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:17.903697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:17.903697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:17.903697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:17.903697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 611/1203] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:17.909068 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c78106-b343-411a-9ce1-725f24da86de None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:18.007344 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c78106-b343-411a-9ce1-725f24da86de None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:18.037010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:18.043694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d85dc5-e20f-439b-be93-89d7fd302b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:18.047267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d85dc5-e20f-439b-be93-89d7fd302b08 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:18.047267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d85dc5-e20f-439b-be93-89d7fd302b08 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:18.047267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d85dc5-e20f-439b-be93-89d7fd302b08 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:18.047267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d85dc5-e20f-439b-be93-89d7fd302b08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:18.047267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d85dc5-e20f-439b-be93-89d7fd302b08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:18.049104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d85dc5-e20f-439b-be93-89d7fd302b08 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:18.060535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 593/1204] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:18.067468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763c787-59be-4656-8191-8450ac8d86e1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:18.138221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:18.184082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763c787-59be-4656-8191-8450ac8d86e1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:18.217074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc2611-035f-4732-a6e9-fcb936cf748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:18.218770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc2611-035f-4732-a6e9-fcb936cf748e None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:18.218770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc2611-035f-4732-a6e9-fcb936cf748e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:18.219020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc2611-035f-4732-a6e9-fcb936cf748e None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:18.219716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc2611-035f-4732-a6e9-fcb936cf748e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:18.221139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc2611-035f-4732-a6e9-fcb936cf748e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:18.331906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 612/1205] 10.222.0.121 () {60 vars in 1137 bytes} [Thu May 26 17:43:18 2022] POST /identity/v3/users => generated 252 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:43:20.591166 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed13301-7cdb-4353-bb9b-a45be2c168e9 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:20.591724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed13301-7cdb-4353-bb9b-a45be2c168e9 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:20.591724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed13301-7cdb-4353-bb9b-a45be2c168e9 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:20.593739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 594/1206] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:43:20 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:43:20.599084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cf1ab-a848-4247-895b-b233ae08627c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:20.602173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cf1ab-a848-4247-895b-b233ae08627c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:20.602173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cf1ab-a848-4247-895b-b233ae08627c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:20.617166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1b5cf1ab-a848-4247-895b-b233ae08627c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:43:20.618355 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1b5cf1ab-a848-4247-895b-b233ae08627c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:43:20.686658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 613/1207] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:43:20 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:43:20.713275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed13301-7cdb-4353-bb9b-a45be2c168e9 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:20.817761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed13301-7cdb-4353-bb9b-a45be2c168e9 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:20.851921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b00ed9-c8b5-48fb-85e7-8a82f0a184b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:20.852639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b00ed9-c8b5-48fb-85e7-8a82f0a184b4 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:20.852916 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b00ed9-c8b5-48fb-85e7-8a82f0a184b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:20.853182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b00ed9-c8b5-48fb-85e7-8a82f0a184b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:20.863700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b00ed9-c8b5-48fb-85e7-8a82f0a184b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-e9b00ed9-c8b5-48fb-85e7-8a82f0a184b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:43:20.871492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:20.872607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:20.873544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 595/1208] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:43:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:20.878914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cf1ab-a848-4247-895b-b233ae08627c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:20.919060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:20.979740 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cf1ab-a848-4247-895b-b233ae08627c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:21.017402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93575508-994c-43e3-b192-5321a3ac9d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:21.018073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93575508-994c-43e3-b192-5321a3ac9d28 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:21.018373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93575508-994c-43e3-b192-5321a3ac9d28 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:21.018629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93575508-994c-43e3-b192-5321a3ac9d28 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:21.019404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93575508-994c-43e3-b192-5321a3ac9d28 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:21.019708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93575508-994c-43e3-b192-5321a3ac9d28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:21.021397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93575508-994c-43e3-b192-5321a3ac9d28 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:21.029267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 614/1209] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:43:20 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:21.036047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b00ed9-c8b5-48fb-85e7-8a82f0a184b4 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:21.087366 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:21.138582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b00ed9-c8b5-48fb-85e7-8a82f0a184b4 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:21.173873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c2fc9-ccdb-42de-91a3-c88d3465d34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:21.174729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c2fc9-ccdb-42de-91a3-c88d3465d34d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:21.175088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c2fc9-ccdb-42de-91a3-c88d3465d34d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:21.175399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c2fc9-ccdb-42de-91a3-c88d3465d34d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:21.187188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c2fc9-ccdb-42de-91a3-c88d3465d34d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:21.188634 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c2fc9-ccdb-42de-91a3-c88d3465d34d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-508c2fc9-ccdb-42de-91a3-c88d3465d34d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:21.206040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:21.206957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:21.207881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:21.207881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:21.207881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:21.207881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:21.207881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 596/1210] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:21.213232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93575508-994c-43e3-b192-5321a3ac9d28 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:21.309166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:21.312154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93575508-994c-43e3-b192-5321a3ac9d28 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:21.346585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9eb70-8797-4126-bebf-50401a980964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:21.347278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9eb70-8797-4126-bebf-50401a980964 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:21.347563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9eb70-8797-4126-bebf-50401a980964 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:21.347839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9eb70-8797-4126-bebf-50401a980964 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:21.348554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9eb70-8797-4126-bebf-50401a980964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:21.348884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9eb70-8797-4126-bebf-50401a980964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:21.350276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9eb70-8797-4126-bebf-50401a980964 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:21.361374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 615/1211] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:21.367013 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c2fc9-ccdb-42de-91a3-c88d3465d34d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:21.468612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c2fc9-ccdb-42de-91a3-c88d3465d34d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:21.470117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:21.502217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896c0c4-9e66-4d3f-8af4-6b4a8763ed2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:21.503038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896c0c4-9e66-4d3f-8af4-6b4a8763ed2d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:21.503316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896c0c4-9e66-4d3f-8af4-6b4a8763ed2d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:21.503586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896c0c4-9e66-4d3f-8af4-6b4a8763ed2d None admin] PATH_INFO: `/v3/users/placement` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:21.510334 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896c0c4-9e66-4d3f-8af4-6b4a8763ed2d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:21.511992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896c0c4-9e66-4d3f-8af4-6b4a8763ed2d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-b896c0c4-9e66-4d3f-8af4-6b4a8763ed2d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:43:21.527506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:21.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:21.529771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:21.529771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:21.529771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:43:21.529771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:43:21.529771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:43:21.529771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:43:21.529771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:43:21.529771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:43:21.529771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 17:43:21.529771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:21.529771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 597/1212] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:21.534845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9eb70-8797-4126-bebf-50401a980964 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:21.634688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9eb70-8797-4126-bebf-50401a980964 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:21.672989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb78279-6cb4-4870-96da-21e6435fb9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:21.673650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb78279-6cb4-4870-96da-21e6435fb9fa None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:21.673931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb78279-6cb4-4870-96da-21e6435fb9fa None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:21.674205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb78279-6cb4-4870-96da-21e6435fb9fa None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:21.675063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb78279-6cb4-4870-96da-21e6435fb9fa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:21.675372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb78279-6cb4-4870-96da-21e6435fb9fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:21.676807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb78279-6cb4-4870-96da-21e6435fb9fa None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:21.694691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 616/1213] 10.222.0.121 () {56 vars in 1148 bytes} [Thu May 26 17:43:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:21.700454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896c0c4-9e66-4d3f-8af4-6b4a8763ed2d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:21.800221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896c0c4-9e66-4d3f-8af4-6b4a8763ed2d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:21.835447 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9f23f-b724-458e-8e27-117479a2c6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:21.837014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9f23f-b724-458e-8e27-117479a2c6ee None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:21.837014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9f23f-b724-458e-8e27-117479a2c6ee None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:21.837014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9f23f-b724-458e-8e27-117479a2c6ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:21.866498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:21.905348 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9f23f-b724-458e-8e27-117479a2c6ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:21.906880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9f23f-b724-458e-8e27-117479a2c6ee None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:22.006055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 598/1214] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:43:21 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:43:22.011995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb78279-6cb4-4870-96da-21e6435fb9fa None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:22.032010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:22.113284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb78279-6cb4-4870-96da-21e6435fb9fa None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:22.147576 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad39b25-8d13-4144-b5c1-733f2c5b7753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:22.148362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad39b25-8d13-4144-b5c1-733f2c5b7753 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:22.148667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad39b25-8d13-4144-b5c1-733f2c5b7753 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:22.148946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad39b25-8d13-4144-b5c1-733f2c5b7753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:22.158948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad39b25-8d13-4144-b5c1-733f2c5b7753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:22.160468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad39b25-8d13-4144-b5c1-733f2c5b7753 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-2ad39b25-8d13-4144-b5c1-733f2c5b7753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:22.178483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:22.179763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:22.180842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:22.180842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:22.180842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:22.180842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:22.180842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 617/1215] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:22.185399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9f23f-b724-458e-8e27-117479a2c6ee None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:22.248416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:22.287017 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9f23f-b724-458e-8e27-117479a2c6ee None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:22.321174 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e22fc-06ab-4173-8713-111bde23ed58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:22.321841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e22fc-06ab-4173-8713-111bde23ed58 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:22.322129 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e22fc-06ab-4173-8713-111bde23ed58 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:22.322401 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e22fc-06ab-4173-8713-111bde23ed58 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:22.323165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e22fc-06ab-4173-8713-111bde23ed58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:22.323454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e22fc-06ab-4173-8713-111bde23ed58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:22.324815 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e22fc-06ab-4173-8713-111bde23ed58 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:22.337093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 599/1216] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:22.342957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad39b25-8d13-4144-b5c1-733f2c5b7753 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:22.415596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:22.448395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad39b25-8d13-4144-b5c1-733f2c5b7753 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:22.482764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7163e4c-1202-493e-8c0a-12a267ed2069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:22.483682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7163e4c-1202-493e-8c0a-12a267ed2069 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:22.484469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7163e4c-1202-493e-8c0a-12a267ed2069 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:22.484735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7163e4c-1202-493e-8c0a-12a267ed2069 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:22.491552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7163e4c-1202-493e-8c0a-12a267ed2069 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:22.492993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7163e4c-1202-493e-8c0a-12a267ed2069 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e7163e4c-1202-493e-8c0a-12a267ed2069 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:43:22.507926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:22.508898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:22.509818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:43:22.509818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:43:22.509818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:43:22.509818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:43:22.509818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:43:22.509818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:43:22.509818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:43:22.509818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:22.509818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 618/1217] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:22.514986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e22fc-06ab-4173-8713-111bde23ed58 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:22.619429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e22fc-06ab-4173-8713-111bde23ed58 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:22.642139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:22.653802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f687e7-acc8-4242-99b1-10166f61c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:22.654472 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f687e7-acc8-4242-99b1-10166f61c88d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:22.654763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f687e7-acc8-4242-99b1-10166f61c88d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:22.655064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f687e7-acc8-4242-99b1-10166f61c88d None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:22.656385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f687e7-acc8-4242-99b1-10166f61c88d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:22.656694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f687e7-acc8-4242-99b1-10166f61c88d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:22.658131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f687e7-acc8-4242-99b1-10166f61c88d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:22.669941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 600/1218] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:43:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:22.677166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7163e4c-1202-493e-8c0a-12a267ed2069 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:22.778347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7163e4c-1202-493e-8c0a-12a267ed2069 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:22.808593 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:22.815881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784340a5-9c33-4add-9c9a-1668131f4abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:22.816536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784340a5-9c33-4add-9c9a-1668131f4abe None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:22.816821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784340a5-9c33-4add-9c9a-1668131f4abe None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:22.817093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784340a5-9c33-4add-9c9a-1668131f4abe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:22.817971 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784340a5-9c33-4add-9c9a-1668131f4abe None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=7847d2c119474d49bef6029d51dbefdb) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:22.818258 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784340a5-9c33-4add-9c9a-1668131f4abe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:22.819721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784340a5-9c33-4add-9c9a-1668131f4abe None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:22.825353 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 619/1219] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:43:22 2022] GET /identity/v3/role_assignments?user.id=7847d2c119474d49bef6029d51dbefdb&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 260 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:25.035474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78d085-bdeb-424e-93e6-7e00b39bf942 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:25.036061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78d085-bdeb-424e-93e6-7e00b39bf942 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:25.036061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78d085-bdeb-424e-93e6-7e00b39bf942 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:25.037379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 601/1220] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:43:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:43:25.043243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a710d20-3f98-4bce-83c0-62d04064d0d5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:25.043686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a710d20-3f98-4bce-83c0-62d04064d0d5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:25.043794 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a710d20-3f98-4bce-83c0-62d04064d0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:25.061483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5a710d20-3f98-4bce-83c0-62d04064d0d5 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:43:25.062791 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5a710d20-3f98-4bce-83c0-62d04064d0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:43:25.130923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 620/1221] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:43:25 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:43:25.153995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78d085-bdeb-424e-93e6-7e00b39bf942 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:25.256036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78d085-bdeb-424e-93e6-7e00b39bf942 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:25.292235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5506c-b29c-43e8-8e25-e99e4a41e0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:25.292984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5506c-b29c-43e8-8e25-e99e4a41e0ab None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:25.293264 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5506c-b29c-43e8-8e25-e99e4a41e0ab None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:25.293539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5506c-b29c-43e8-8e25-e99e4a41e0ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:25.303987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5506c-b29c-43e8-8e25-e99e4a41e0ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-44a5506c-b29c-43e8-8e25-e99e4a41e0ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:43:25.311660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:25.312822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:25.313812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 602/1222] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:43:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:25.319219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a710d20-3f98-4bce-83c0-62d04064d0d5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:25.417450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a710d20-3f98-4bce-83c0-62d04064d0d5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:25.440322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:25.451066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffc5f2-d5b6-4a27-8d84-a5ab6561d950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:25.451732 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffc5f2-d5b6-4a27-8d84-a5ab6561d950 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:25.452009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffc5f2-d5b6-4a27-8d84-a5ab6561d950 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:25.452277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffc5f2-d5b6-4a27-8d84-a5ab6561d950 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:25.453632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffc5f2-d5b6-4a27-8d84-a5ab6561d950 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:25.453928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffc5f2-d5b6-4a27-8d84-a5ab6561d950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:25.455310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffc5f2-d5b6-4a27-8d84-a5ab6561d950 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:25.463225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 621/1223] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:43:25 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:25.469845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5506c-b29c-43e8-8e25-e99e4a41e0ab None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:25.514317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:25.571317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5506c-b29c-43e8-8e25-e99e4a41e0ab None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:25.605878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905f3cc-805b-4c09-8774-b651a1556d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:25.606665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905f3cc-805b-4c09-8774-b651a1556d86 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:25.606988 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905f3cc-805b-4c09-8774-b651a1556d86 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:25.607263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905f3cc-805b-4c09-8774-b651a1556d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:25.618647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905f3cc-805b-4c09-8774-b651a1556d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:25.620123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905f3cc-805b-4c09-8774-b651a1556d86 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-b905f3cc-805b-4c09-8774-b651a1556d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:25.636663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:25.637746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:25.638863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:25.638863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:25.638863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:25.638863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:25.638863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 603/1224] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:25.643872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffc5f2-d5b6-4a27-8d84-a5ab6561d950 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:25.745341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffc5f2-d5b6-4a27-8d84-a5ab6561d950 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:25.778762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c124b9-e34b-46ea-b58a-59fd5756ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:25.779442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c124b9-e34b-46ea-b58a-59fd5756ce5d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:25.779720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c124b9-e34b-46ea-b58a-59fd5756ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:25.779990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c124b9-e34b-46ea-b58a-59fd5756ce5d None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:25.781266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c124b9-e34b-46ea-b58a-59fd5756ce5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:25.781603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c124b9-e34b-46ea-b58a-59fd5756ce5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:25.783074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c124b9-e34b-46ea-b58a-59fd5756ce5d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:25.794957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 622/1225] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:25.801436 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905f3cc-805b-4c09-8774-b651a1556d86 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:25.894548 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:25.901071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905f3cc-805b-4c09-8774-b651a1556d86 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:25.936201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec579f-2860-4ba2-bf7b-6154b46c975a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:25.937082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec579f-2860-4ba2-bf7b-6154b46c975a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:25.937398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec579f-2860-4ba2-bf7b-6154b46c975a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:25.937678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec579f-2860-4ba2-bf7b-6154b46c975a None admin] PATH_INFO: `/v3/users/placement` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:25.944966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec579f-2860-4ba2-bf7b-6154b46c975a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:25.946451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec579f-2860-4ba2-bf7b-6154b46c975a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-2cec579f-2860-4ba2-bf7b-6154b46c975a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:43:25.961784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:25.962977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:25.964333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:25.964333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:25.964333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:43:25.964333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:43:25.964333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:43:25.964333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:43:25.964333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:43:25.964333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:43:25.964333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 17:43:25.964333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:25.964333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 604/1226] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:25.969888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c124b9-e34b-46ea-b58a-59fd5756ce5d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:26.019208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:26.073343 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c124b9-e34b-46ea-b58a-59fd5756ce5d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:26.107557 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58f401-2de7-45ff-b780-2ebb9cbd3907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:26.107788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58f401-2de7-45ff-b780-2ebb9cbd3907 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:26.108064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58f401-2de7-45ff-b780-2ebb9cbd3907 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:26.108332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58f401-2de7-45ff-b780-2ebb9cbd3907 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:26.109672 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58f401-2de7-45ff-b780-2ebb9cbd3907 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:26.109972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58f401-2de7-45ff-b780-2ebb9cbd3907 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:26.111475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58f401-2de7-45ff-b780-2ebb9cbd3907 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:26.132019 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 623/1227] 10.222.0.121 () {56 vars in 1148 bytes} [Thu May 26 17:43:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:43:26.138724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec579f-2860-4ba2-bf7b-6154b46c975a None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:26.236036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec579f-2860-4ba2-bf7b-6154b46c975a None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:26.270976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cbf1c-bd1d-4783-b387-2773d8e91570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:26.271784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cbf1c-bd1d-4783-b387-2773d8e91570 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:26.272063 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cbf1c-bd1d-4783-b387-2773d8e91570 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:26.272335 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cbf1c-bd1d-4783-b387-2773d8e91570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:26.283790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cbf1c-bd1d-4783-b387-2773d8e91570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:26.285219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cbf1c-bd1d-4783-b387-2773d8e91570 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-533cbf1c-bd1d-4783-b387-2773d8e91570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:26.303395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:26.305573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:26.306494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:26.306494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:26.306494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:26.306494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:26.306494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 605/1228] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:26.310382 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58f401-2de7-45ff-b780-2ebb9cbd3907 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:26.410389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58f401-2de7-45ff-b780-2ebb9cbd3907 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:26.413178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:26.444667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b802e-7bba-4f22-8bea-c40cc6743c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:26.445331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b802e-7bba-4f22-8bea-c40cc6743c05 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:26.445616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b802e-7bba-4f22-8bea-c40cc6743c05 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:26.445890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b802e-7bba-4f22-8bea-c40cc6743c05 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:26.446601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b802e-7bba-4f22-8bea-c40cc6743c05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:26.446938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b802e-7bba-4f22-8bea-c40cc6743c05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:26.448321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b802e-7bba-4f22-8bea-c40cc6743c05 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:26.459472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 624/1229] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:26.465142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cbf1c-bd1d-4783-b387-2773d8e91570 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:26.474684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:26.565617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cbf1c-bd1d-4783-b387-2773d8e91570 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:26.598952 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e560f-6369-48f3-9a4a-4a29c86bced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:26.599742 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e560f-6369-48f3-9a4a-4a29c86bced7 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:26.600018 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e560f-6369-48f3-9a4a-4a29c86bced7 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:26.600284 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e560f-6369-48f3-9a4a-4a29c86bced7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:26.606889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e560f-6369-48f3-9a4a-4a29c86bced7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:26.609730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e560f-6369-48f3-9a4a-4a29c86bced7 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-4d7e560f-6369-48f3-9a4a-4a29c86bced7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:43:26.623724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:26.624844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:26.626058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:43:26.626058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:43:26.626058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:43:26.626058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:43:26.626058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:43:26.626058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:43:26.626058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:43:26.626058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:26.626058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 606/1230] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:26.631097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b802e-7bba-4f22-8bea-c40cc6743c05 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:26.732585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b802e-7bba-4f22-8bea-c40cc6743c05 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:26.767060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b4aae-8167-43e5-8bc5-c2b1325fccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:26.767720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b4aae-8167-43e5-8bc5-c2b1325fccee None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:26.767997 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b4aae-8167-43e5-8bc5-c2b1325fccee None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:26.768266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b4aae-8167-43e5-8bc5-c2b1325fccee None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:26.769226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b4aae-8167-43e5-8bc5-c2b1325fccee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:26.769628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b4aae-8167-43e5-8bc5-c2b1325fccee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:26.770985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b4aae-8167-43e5-8bc5-c2b1325fccee None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:26.791621 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 625/1231] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:43:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:26.798454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e560f-6369-48f3-9a4a-4a29c86bced7 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:26.866199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:26.900686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e560f-6369-48f3-9a4a-4a29c86bced7 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:26.938064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd41dc-0bb4-4cee-bf8e-e146ecabbbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:26.939081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd41dc-0bb4-4cee-bf8e-e146ecabbbb8 None admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:26.939363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd41dc-0bb4-4cee-bf8e-e146ecabbbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:26.939642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd41dc-0bb4-4cee-bf8e-e146ecabbbb8 None admin] PATH_INFO: `/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/7847d2c119474d49bef6029d51dbefdb/roles/9399fac5b71b46f79d1ebe5142fc1176` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:26.956557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd41dc-0bb4-4cee-bf8e-e146ecabbbb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61dea65b34b47a186151c6a63bc7d79, user_id=7847d2c119474d49bef6029d51dbefdb, role_id=9399fac5b71b46f79d1ebe5142fc1176)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:26.958162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd41dc-0bb4-4cee-bf8e-e146ecabbbb8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:27.121854 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 607/1232] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:43:26 2022] PUT /identity/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/7847d2c119474d49bef6029d51dbefdb/roles/9399fac5b71b46f79d1ebe5142fc1176 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:43:29.377003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729541c3-2d9a-488b-84da-9dc25faa2289 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:29.377651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729541c3-2d9a-488b-84da-9dc25faa2289 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:29.377651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729541c3-2d9a-488b-84da-9dc25faa2289 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:29.379113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 626/1233] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:43:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:43:29.384587 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64596196-7b8a-4469-bd7a-94fa9092cb7b None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:29.385015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64596196-7b8a-4469-bd7a-94fa9092cb7b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:29.385107 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64596196-7b8a-4469-bd7a-94fa9092cb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:29.401957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-64596196-7b8a-4469-bd7a-94fa9092cb7b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:43:29.403170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-64596196-7b8a-4469-bd7a-94fa9092cb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:43:29.469862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 608/1234] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:43:29 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:43:29.494452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729541c3-2d9a-488b-84da-9dc25faa2289 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:29.519439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:29.595376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729541c3-2d9a-488b-84da-9dc25faa2289 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:29.629848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c0b3b-9e87-477a-91a3-3711fc24c226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:29.630568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c0b3b-9e87-477a-91a3-3711fc24c226 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:29.630868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c0b3b-9e87-477a-91a3-3711fc24c226 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:29.631144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c0b3b-9e87-477a-91a3-3711fc24c226 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:29.642687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c0b3b-9e87-477a-91a3-3711fc24c226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-889c0b3b-9e87-477a-91a3-3711fc24c226 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:43:29.650570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:29.651801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:29.653000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 627/1235] 10.222.0.121 () {56 vars in 1107 bytes} [Thu May 26 17:43:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:29.658463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64596196-7b8a-4469-bd7a-94fa9092cb7b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:29.704807 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:29.756321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64596196-7b8a-4469-bd7a-94fa9092cb7b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:29.792682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fa5ee-a79f-4348-887b-b1af3b10f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:29.793350 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fa5ee-a79f-4348-887b-b1af3b10f620 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:29.793646 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fa5ee-a79f-4348-887b-b1af3b10f620 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:29.793903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fa5ee-a79f-4348-887b-b1af3b10f620 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:29.794686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fa5ee-a79f-4348-887b-b1af3b10f620 None admin] RBAC: Adding query filter params (name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:29.795015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fa5ee-a79f-4348-887b-b1af3b10f620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:29.796821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fa5ee-a79f-4348-887b-b1af3b10f620 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:29.804877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 609/1236] 10.222.0.121 () {56 vars in 1108 bytes} [Thu May 26 17:43:29 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:29.810754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c0b3b-9e87-477a-91a3-3711fc24c226 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:29.896718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:29.912410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c0b3b-9e87-477a-91a3-3711fc24c226 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:29.945713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4557b-6dda-4c94-852c-fd5f3265d9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:29.946508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4557b-6dda-4c94-852c-fd5f3265d9ca None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:29.946787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4557b-6dda-4c94-852c-fd5f3265d9ca None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:29.947091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4557b-6dda-4c94-852c-fd5f3265d9ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:29.959090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4557b-6dda-4c94-852c-fd5f3265d9ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:29.960568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4557b-6dda-4c94-852c-fd5f3265d9ca None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-83c4557b-6dda-4c94-852c-fd5f3265d9ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:29.979982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:29.981135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:29.987484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:29.987484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:29.987484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:29.987484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:29.987484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 628/1237] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:29.988034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fa5ee-a79f-4348-887b-b1af3b10f620 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:30.111087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fa5ee-a79f-4348-887b-b1af3b10f620 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:30.124955 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:30.151920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3b499-5d73-4aa6-9af9-98ed2d279351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:30.152520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3b499-5d73-4aa6-9af9-98ed2d279351 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:30.152802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3b499-5d73-4aa6-9af9-98ed2d279351 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:30.153082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3b499-5d73-4aa6-9af9-98ed2d279351 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:30.153801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3b499-5d73-4aa6-9af9-98ed2d279351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:30.154158 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3b499-5d73-4aa6-9af9-98ed2d279351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:30.155639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3b499-5d73-4aa6-9af9-98ed2d279351 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:30.168124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 610/1238] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:30.174289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4557b-6dda-4c94-852c-fd5f3265d9ca None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:30.276773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4557b-6dda-4c94-852c-fd5f3265d9ca None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:30.311005 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c04b2-4efe-430c-91c3-20be58449404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:30.312477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c04b2-4efe-430c-91c3-20be58449404 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:30.312759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c04b2-4efe-430c-91c3-20be58449404 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:30.313124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c04b2-4efe-430c-91c3-20be58449404 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:30.315914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:30.320491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c04b2-4efe-430c-91c3-20be58449404 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:30.321938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c04b2-4efe-430c-91c3-20be58449404 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-6f2c04b2-4efe-430c-91c3-20be58449404 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:43:30.338636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:30.339793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:30.340984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:30.340984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:30.340984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:43:30.340984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:43:30.340984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:43:30.340984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:43:30.340984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:43:30.340984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:43:30.340984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 17:43:30.340984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:30.340984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 629/1239] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:30.345791 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3b499-5d73-4aa6-9af9-98ed2d279351 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:30.452457 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3b499-5d73-4aa6-9af9-98ed2d279351 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:30.487265 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba7d8d-c15c-4927-8697-5f1d7c3cdd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:30.487940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba7d8d-c15c-4927-8697-5f1d7c3cdd20 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:30.488219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba7d8d-c15c-4927-8697-5f1d7c3cdd20 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:30.488487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba7d8d-c15c-4927-8697-5f1d7c3cdd20 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:30.489307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba7d8d-c15c-4927-8697-5f1d7c3cdd20 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:30.489650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba7d8d-c15c-4927-8697-5f1d7c3cdd20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:30.491303 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba7d8d-c15c-4927-8697-5f1d7c3cdd20 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:30.509136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 611/1240] 10.222.0.121 () {56 vars in 1148 bytes} [Thu May 26 17:43:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:43:30.515154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c04b2-4efe-430c-91c3-20be58449404 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:30.624886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c04b2-4efe-430c-91c3-20be58449404 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:30.658881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4696ba-0fd0-4525-ba62-150b197c3c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:30.659610 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4696ba-0fd0-4525-ba62-150b197c3c03 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:30.659830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4696ba-0fd0-4525-ba62-150b197c3c03 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:30.660109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4696ba-0fd0-4525-ba62-150b197c3c03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:30.725199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:30.731603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4696ba-0fd0-4525-ba62-150b197c3c03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:30.733065 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4696ba-0fd0-4525-ba62-150b197c3c03 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:30.840066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 630/1241] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:43:30 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:43:30.847050 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba7d8d-c15c-4927-8697-5f1d7c3cdd20 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:30.875517 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:30.946554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba7d8d-c15c-4927-8697-5f1d7c3cdd20 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:30.979890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbbb28-1229-475c-9340-8fbb655f7082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:30.980619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbbb28-1229-475c-9340-8fbb655f7082 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:30.980900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbbb28-1229-475c-9340-8fbb655f7082 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:30.981226 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbbb28-1229-475c-9340-8fbb655f7082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:30.996652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbbb28-1229-475c-9340-8fbb655f7082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:30.998434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbbb28-1229-475c-9340-8fbb655f7082 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-00dbbb28-1229-475c-9340-8fbb655f7082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:31.017829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:31.018946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:31.020236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:31.020236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:31.020236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:31.020236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:31.020236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 612/1242] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:31.024874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4696ba-0fd0-4525-ba62-150b197c3c03 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:31.136435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:31.143086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4696ba-0fd0-4525-ba62-150b197c3c03 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:31.190751 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09407924-e13f-48b1-b24b-da93d991b24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:31.191466 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09407924-e13f-48b1-b24b-da93d991b24b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:31.191748 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09407924-e13f-48b1-b24b-da93d991b24b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:31.192023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09407924-e13f-48b1-b24b-da93d991b24b None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:31.192760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09407924-e13f-48b1-b24b-da93d991b24b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:31.193068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09407924-e13f-48b1-b24b-da93d991b24b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:31.194446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09407924-e13f-48b1-b24b-da93d991b24b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:31.206781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 631/1243] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:31.212801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbbb28-1229-475c-9340-8fbb655f7082 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:31.300279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:31.323081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbbb28-1229-475c-9340-8fbb655f7082 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:31.356210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e84c81-8b87-46c6-b764-b980f3dc4774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:31.357002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e84c81-8b87-46c6-b764-b980f3dc4774 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:31.357287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e84c81-8b87-46c6-b764-b980f3dc4774 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:31.357560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e84c81-8b87-46c6-b764-b980f3dc4774 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:31.365821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e84c81-8b87-46c6-b764-b980f3dc4774 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:31.367394 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e84c81-8b87-46c6-b764-b980f3dc4774 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d1e84c81-8b87-46c6-b764-b980f3dc4774 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:43:31.381697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:31.390602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:31.391843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:43:31.391843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:43:31.391843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:43:31.391843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:43:31.391843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:43:31.391843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:43:31.391843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:43:31.391843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:31.391843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 613/1244] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:43:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:43:31.392485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09407924-e13f-48b1-b24b-da93d991b24b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:31.492362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09407924-e13f-48b1-b24b-da93d991b24b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:31.529489 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6ad13-069a-47be-9987-b8d26659f9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:31.530663 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6ad13-069a-47be-9987-b8d26659f9e9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:31.531123 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6ad13-069a-47be-9987-b8d26659f9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:31.531395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6ad13-069a-47be-9987-b8d26659f9e9 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:31.532172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6ad13-069a-47be-9987-b8d26659f9e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:31.532476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6ad13-069a-47be-9987-b8d26659f9e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:31.533907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6ad13-069a-47be-9987-b8d26659f9e9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:31.547632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 632/1245] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:43:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:31.553980 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e84c81-8b87-46c6-b764-b980f3dc4774 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:31.662112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e84c81-8b87-46c6-b764-b980f3dc4774 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:31.696957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6c219-5e40-42ef-9f02-5513744992c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:31.697669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6c219-5e40-42ef-9f02-5513744992c0 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:31.697891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6c219-5e40-42ef-9f02-5513744992c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:31.698160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6c219-5e40-42ef-9f02-5513744992c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:31.699009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6c219-5e40-42ef-9f02-5513744992c0 None admin] RBAC: Adding query filter params (role.id=9399fac5b71b46f79d1ebe5142fc1176, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=7847d2c119474d49bef6029d51dbefdb) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:31.699320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6c219-5e40-42ef-9f02-5513744992c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:31.700748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6c219-5e40-42ef-9f02-5513744992c0 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:31.706664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 614/1246] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:43:31 2022] GET /identity/v3/role_assignments?user.id=7847d2c119474d49bef6029d51dbefdb&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=9399fac5b71b46f79d1ebe5142fc1176 => generated 610 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:34.035913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c95de0-24e7-477c-871e-2993de473be2 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:34.036574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c95de0-24e7-477c-871e-2993de473be2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:34.036574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c95de0-24e7-477c-871e-2993de473be2 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:34.038153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 633/1247] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:43:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:43:34.043318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594a8a8-c1b4-4e97-8348-be74296797f5 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:34.043741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594a8a8-c1b4-4e97-8348-be74296797f5 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:34.043853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594a8a8-c1b4-4e97-8348-be74296797f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:34.049297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:34.061156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c594a8a8-c1b4-4e97-8348-be74296797f5 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:43:34.062965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c594a8a8-c1b4-4e97-8348-be74296797f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:43:34.132250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 615/1248] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:43:34 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:43:34.156704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c95de0-24e7-477c-871e-2993de473be2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:34.176615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:34.260850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c95de0-24e7-477c-871e-2993de473be2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:34.294204 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2f1fe-6ead-476c-aa6b-7172088e3630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:34.294950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2f1fe-6ead-476c-aa6b-7172088e3630 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:34.295243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2f1fe-6ead-476c-aa6b-7172088e3630 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:34.295498 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2f1fe-6ead-476c-aa6b-7172088e3630 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:34.306504 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2f1fe-6ead-476c-aa6b-7172088e3630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-41d2f1fe-6ead-476c-aa6b-7172088e3630 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:43:34.314788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:34.316085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:34.317320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 634/1249] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:43:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:34.322172 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594a8a8-c1b4-4e97-8348-be74296797f5 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:34.422107 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594a8a8-c1b4-4e97-8348-be74296797f5 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:34.450520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:34.455478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec244de-851d-40a1-b295-6058b3f39e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:34.456141 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec244de-851d-40a1-b295-6058b3f39e67 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:34.456416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec244de-851d-40a1-b295-6058b3f39e67 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:34.456684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec244de-851d-40a1-b295-6058b3f39e67 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:34.457417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec244de-851d-40a1-b295-6058b3f39e67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:34.457716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec244de-851d-40a1-b295-6058b3f39e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:34.459109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec244de-851d-40a1-b295-6058b3f39e67 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:34.467027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 616/1250] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:43:34 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:34.473163 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2f1fe-6ead-476c-aa6b-7172088e3630 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:34.554240 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:34.575565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2f1fe-6ead-476c-aa6b-7172088e3630 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:34.609164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad5af9-f122-4a6c-8e77-bf38d01aafd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:34.609989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad5af9-f122-4a6c-8e77-bf38d01aafd7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:34.610277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad5af9-f122-4a6c-8e77-bf38d01aafd7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:34.610538 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad5af9-f122-4a6c-8e77-bf38d01aafd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:34.621644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad5af9-f122-4a6c-8e77-bf38d01aafd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:34.623169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad5af9-f122-4a6c-8e77-bf38d01aafd7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e0ad5af9-f122-4a6c-8e77-bf38d01aafd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:34.640836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:34.641927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:34.647378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:34.647378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:34.647378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:34.647378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:34.647378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 635/1251] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:34.647796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec244de-851d-40a1-b295-6058b3f39e67 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:34.747987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec244de-851d-40a1-b295-6058b3f39e67 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:34.782427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0238e04-da16-456d-bc1a-3e0b60ce8a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:34.784342 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0238e04-da16-456d-bc1a-3e0b60ce8a9f None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:34.784342 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0238e04-da16-456d-bc1a-3e0b60ce8a9f None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:34.784342 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0238e04-da16-456d-bc1a-3e0b60ce8a9f None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:34.784609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0238e04-da16-456d-bc1a-3e0b60ce8a9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:34.784700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0238e04-da16-456d-bc1a-3e0b60ce8a9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:34.786030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0238e04-da16-456d-bc1a-3e0b60ce8a9f None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:34.798494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 617/1252] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:34.807338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad5af9-f122-4a6c-8e77-bf38d01aafd7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:34.911837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad5af9-f122-4a6c-8e77-bf38d01aafd7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:34.939646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:34.947323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd367180-b331-4694-9ce8-22666bd99e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:34.948121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd367180-b331-4694-9ce8-22666bd99e7c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:34.948399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd367180-b331-4694-9ce8-22666bd99e7c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:34.948670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd367180-b331-4694-9ce8-22666bd99e7c None admin] PATH_INFO: `/v3/users/placement` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:34.956121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd367180-b331-4694-9ce8-22666bd99e7c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:34.957661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd367180-b331-4694-9ce8-22666bd99e7c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-cd367180-b331-4694-9ce8-22666bd99e7c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:43:34.971893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:34.972866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:34.974010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:34.974010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:34.974010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:43:34.974010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:43:34.974010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:43:34.974010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:43:34.974010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:43:34.974010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:43:34.974010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 17:43:34.974010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:34.974010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 636/1253] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:34.979015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0238e04-da16-456d-bc1a-3e0b60ce8a9f None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:35.035642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:35.081064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0238e04-da16-456d-bc1a-3e0b60ce8a9f None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:35.114919 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8288a-6564-4a7c-a0ce-066eb7354d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:35.115590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8288a-6564-4a7c-a0ce-066eb7354d75 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:35.115871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8288a-6564-4a7c-a0ce-066eb7354d75 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:35.116139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8288a-6564-4a7c-a0ce-066eb7354d75 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:35.117041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8288a-6564-4a7c-a0ce-066eb7354d75 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:35.117339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8288a-6564-4a7c-a0ce-066eb7354d75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:35.118788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8288a-6564-4a7c-a0ce-066eb7354d75 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:35.137612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 618/1254] 10.222.0.121 () {56 vars in 1148 bytes} [Thu May 26 17:43:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:35.144117 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd367180-b331-4694-9ce8-22666bd99e7c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:35.244873 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd367180-b331-4694-9ce8-22666bd99e7c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:35.277859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04e531-1ab1-4bf0-8d9c-4426d54d4093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:35.278498 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04e531-1ab1-4bf0-8d9c-4426d54d4093 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:35.278775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04e531-1ab1-4bf0-8d9c-4426d54d4093 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:35.279071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04e531-1ab1-4bf0-8d9c-4426d54d4093 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:35.333534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:35.345996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04e531-1ab1-4bf0-8d9c-4426d54d4093 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:35.347517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04e531-1ab1-4bf0-8d9c-4426d54d4093 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:35.446061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 637/1255] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:43:35 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:43:35.451973 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8288a-6564-4a7c-a0ce-066eb7354d75 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:35.553034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8288a-6564-4a7c-a0ce-066eb7354d75 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:35.566308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:35.587142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44d002-0792-4771-9109-386fb80d04d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:35.587836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44d002-0792-4771-9109-386fb80d04d8 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:35.588113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44d002-0792-4771-9109-386fb80d04d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:35.588489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44d002-0792-4771-9109-386fb80d04d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:35.599036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44d002-0792-4771-9109-386fb80d04d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:35.600461 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44d002-0792-4771-9109-386fb80d04d8 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-ce44d002-0792-4771-9109-386fb80d04d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:35.618024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:35.619150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:35.620138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:35.620138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:35.620138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:35.620138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:35.620138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 619/1256] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:35.625347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04e531-1ab1-4bf0-8d9c-4426d54d4093 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:35.714112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:35.726734 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04e531-1ab1-4bf0-8d9c-4426d54d4093 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:35.760060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ad891-d55c-4d12-87fc-07a9cb38e26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:35.760735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ad891-d55c-4d12-87fc-07a9cb38e26a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:35.761017 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ad891-d55c-4d12-87fc-07a9cb38e26a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:35.761293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ad891-d55c-4d12-87fc-07a9cb38e26a None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:35.762006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ad891-d55c-4d12-87fc-07a9cb38e26a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:35.762313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ad891-d55c-4d12-87fc-07a9cb38e26a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:35.763765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ad891-d55c-4d12-87fc-07a9cb38e26a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:35.776357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 638/1257] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:35.782921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44d002-0792-4771-9109-386fb80d04d8 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:35.883232 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44d002-0792-4771-9109-386fb80d04d8 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:35.917986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ae9d-7827-44fe-a6ca-dd369373f329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:35.918782 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ae9d-7827-44fe-a6ca-dd369373f329 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:35.919098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ae9d-7827-44fe-a6ca-dd369373f329 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:35.919371 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ae9d-7827-44fe-a6ca-dd369373f329 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:35.926026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9ae9d-7827-44fe-a6ca-dd369373f329 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:35.927496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9ae9d-7827-44fe-a6ca-dd369373f329 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d2b9ae9d-7827-44fe-a6ca-dd369373f329 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:43:35.941333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:35.942319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:35.943279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:43:35.943279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:43:35.943279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:43:35.943279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:43:35.943279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:43:35.943279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:43:35.943279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:43:35.943279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:35.943279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 620/1258] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:43:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:35.948598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ad891-d55c-4d12-87fc-07a9cb38e26a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:36.051288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ad891-d55c-4d12-87fc-07a9cb38e26a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:36.086745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bba992-5e8e-4857-a1da-49f0417c87ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:36.087468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bba992-5e8e-4857-a1da-49f0417c87ad None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:36.087753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bba992-5e8e-4857-a1da-49f0417c87ad None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:36.088032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bba992-5e8e-4857-a1da-49f0417c87ad None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:36.088814 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bba992-5e8e-4857-a1da-49f0417c87ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:36.089122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bba992-5e8e-4857-a1da-49f0417c87ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:36.090552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bba992-5e8e-4857-a1da-49f0417c87ad None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:36.102776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 639/1259] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:43:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:36.109422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ae9d-7827-44fe-a6ca-dd369373f329 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:36.137031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:36.213335 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ae9d-7827-44fe-a6ca-dd369373f329 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:36.249384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c176853-a2c0-4c88-a13f-8b4c0c211a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:36.250048 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c176853-a2c0-4c88-a13f-8b4c0c211a0a None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:36.250309 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c176853-a2c0-4c88-a13f-8b4c0c211a0a None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:36.250583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c176853-a2c0-4c88-a13f-8b4c0c211a0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:36.251438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c176853-a2c0-4c88-a13f-8b4c0c211a0a None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=7847d2c119474d49bef6029d51dbefdb) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:36.251750 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c176853-a2c0-4c88-a13f-8b4c0c211a0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:36.253172 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c176853-a2c0-4c88-a13f-8b4c0c211a0a None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:36.258654 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 621/1260] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:43:36 2022] GET /identity/v3/role_assignments?user.id=7847d2c119474d49bef6029d51dbefdb&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=4f574961798246a4b12ad2276e67685d => generated 260 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:38.475218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2c9d0-b2a5-4b27-813a-8147757c5c41 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:38.475774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2c9d0-b2a5-4b27-813a-8147757c5c41 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:38.475774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2c9d0-b2a5-4b27-813a-8147757c5c41 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:38.477170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 640/1261] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:43:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:43:38.482478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3c34e-b308-4ee9-8a5d-408200942a03 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:38.483040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3c34e-b308-4ee9-8a5d-408200942a03 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:38.483143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3c34e-b308-4ee9-8a5d-408200942a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:38.500390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-39b3c34e-b308-4ee9-8a5d-408200942a03 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:43:38.501657 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-39b3c34e-b308-4ee9-8a5d-408200942a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:43:38.570500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 622/1262] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:43:38 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:43:38.594006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2c9d0-b2a5-4b27-813a-8147757c5c41 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:38.604965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:38.730949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2c9d0-b2a5-4b27-813a-8147757c5c41 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:38.767700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408f73c-ca58-4acb-a9a5-82a678dd633c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:38.768499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408f73c-ca58-4acb-a9a5-82a678dd633c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:38.768786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408f73c-ca58-4acb-a9a5-82a678dd633c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:38.769057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408f73c-ca58-4acb-a9a5-82a678dd633c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:38.779506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408f73c-ca58-4acb-a9a5-82a678dd633c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-1408f73c-ca58-4acb-a9a5-82a678dd633c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:43:38.787924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:38.789227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:38.790129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 641/1263] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:43:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:38.795962 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3c34e-b308-4ee9-8a5d-408200942a03 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:38.815544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:38.895162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3c34e-b308-4ee9-8a5d-408200942a03 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:38.930596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92e3ca-356e-4b6a-841f-261477f42427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:38.931294 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92e3ca-356e-4b6a-841f-261477f42427 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:38.931577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92e3ca-356e-4b6a-841f-261477f42427 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:38.931851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92e3ca-356e-4b6a-841f-261477f42427 None admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:38.932591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92e3ca-356e-4b6a-841f-261477f42427 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:38.932895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92e3ca-356e-4b6a-841f-261477f42427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:38.934319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92e3ca-356e-4b6a-841f-261477f42427 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:38.943663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 623/1264] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:43:38 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:38.949644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408f73c-ca58-4acb-a9a5-82a678dd633c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:39.019082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:39.053573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408f73c-ca58-4acb-a9a5-82a678dd633c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:39.086483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd18fb6-20c7-4996-bed2-0d585acc1a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:39.087298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd18fb6-20c7-4996-bed2-0d585acc1a23 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:39.087594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd18fb6-20c7-4996-bed2-0d585acc1a23 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:39.087888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd18fb6-20c7-4996-bed2-0d585acc1a23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:39.098929 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd18fb6-20c7-4996-bed2-0d585acc1a23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:39.100382 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd18fb6-20c7-4996-bed2-0d585acc1a23 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-ccd18fb6-20c7-4996-bed2-0d585acc1a23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:39.121696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:39.122847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:39.128821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:39.128821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:39.128821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:39.128821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:39.128821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 642/1265] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:39.129263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92e3ca-356e-4b6a-841f-261477f42427 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:39.212470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:39.231848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92e3ca-356e-4b6a-841f-261477f42427 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:39.266093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2ffe6-15e5-41ba-8937-838c085b8669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:39.268482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2ffe6-15e5-41ba-8937-838c085b8669 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:39.268482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2ffe6-15e5-41ba-8937-838c085b8669 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:39.268482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2ffe6-15e5-41ba-8937-838c085b8669 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:39.268482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2ffe6-15e5-41ba-8937-838c085b8669 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:39.268482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2ffe6-15e5-41ba-8937-838c085b8669 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:39.269961 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2ffe6-15e5-41ba-8937-838c085b8669 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:39.282274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 624/1266] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:39.288360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd18fb6-20c7-4996-bed2-0d585acc1a23 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:39.391740 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd18fb6-20c7-4996-bed2-0d585acc1a23 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:39.412000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:39.425552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734f2eb-843a-4bca-b532-38d2fc1bee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:39.426494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734f2eb-843a-4bca-b532-38d2fc1bee2a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:39.426789 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734f2eb-843a-4bca-b532-38d2fc1bee2a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:39.427094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734f2eb-843a-4bca-b532-38d2fc1bee2a None admin] PATH_INFO: `/v3/users/placement` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:39.434634 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734f2eb-843a-4bca-b532-38d2fc1bee2a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:39.436188 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734f2eb-843a-4bca-b532-38d2fc1bee2a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-0734f2eb-843a-4bca-b532-38d2fc1bee2a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:43:39.450537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:39.452267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:39.453257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:39.453257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:39.453257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:43:39.453257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:43:39.453257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:43:39.453257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:43:39.453257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:43:39.453257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:43:39.453257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 17:43:39.453257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:39.453257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 643/1267] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:39.458413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2ffe6-15e5-41ba-8937-838c085b8669 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:39.561497 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2ffe6-15e5-41ba-8937-838c085b8669 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:39.595069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3450a7-cc83-4351-b189-0ce7dddd0049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:39.595738 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3450a7-cc83-4351-b189-0ce7dddd0049 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:39.596015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3450a7-cc83-4351-b189-0ce7dddd0049 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:39.596285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3450a7-cc83-4351-b189-0ce7dddd0049 None admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:39.597156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3450a7-cc83-4351-b189-0ce7dddd0049 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:39.597457 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3450a7-cc83-4351-b189-0ce7dddd0049 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:39.598896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3450a7-cc83-4351-b189-0ce7dddd0049 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:39.608644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:39.616828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 625/1268] 10.222.0.121 () {56 vars in 1148 bytes} [Thu May 26 17:43:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:39.622705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734f2eb-843a-4bca-b532-38d2fc1bee2a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:39.725230 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734f2eb-843a-4bca-b532-38d2fc1bee2a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:39.759301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4266c-a0cd-424d-ae0e-2947a4c2cb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:39.760094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4266c-a0cd-424d-ae0e-2947a4c2cb99 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:39.760369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4266c-a0cd-424d-ae0e-2947a4c2cb99 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:39.760637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4266c-a0cd-424d-ae0e-2947a4c2cb99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:39.771349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4266c-a0cd-424d-ae0e-2947a4c2cb99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:39.775169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4266c-a0cd-424d-ae0e-2947a4c2cb99 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-adc4266c-a0cd-424d-ae0e-2947a4c2cb99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:39.790362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:39.796227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:39.797336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:39.797336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:39.797336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:39.797336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:39.797336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 644/1269] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:39.798179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3450a7-cc83-4351-b189-0ce7dddd0049 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:39.901214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3450a7-cc83-4351-b189-0ce7dddd0049 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:39.936489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6dc82-2ef0-49db-b9de-bfc5443e6398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:39.937239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6dc82-2ef0-49db-b9de-bfc5443e6398 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:39.937522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6dc82-2ef0-49db-b9de-bfc5443e6398 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:39.937798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6dc82-2ef0-49db-b9de-bfc5443e6398 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:39.938518 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6dc82-2ef0-49db-b9de-bfc5443e6398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:39.938852 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6dc82-2ef0-49db-b9de-bfc5443e6398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:39.940223 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6dc82-2ef0-49db-b9de-bfc5443e6398 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:39.951881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 626/1270] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:39.957800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4266c-a0cd-424d-ae0e-2947a4c2cb99 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:39.993821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:40.063710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4266c-a0cd-424d-ae0e-2947a4c2cb99 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:40.098419 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01c6b8-57bb-4776-8fc4-098688b08fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:40.099262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01c6b8-57bb-4776-8fc4-098688b08fcb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:40.099544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01c6b8-57bb-4776-8fc4-098688b08fcb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:40.099816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01c6b8-57bb-4776-8fc4-098688b08fcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:40.108688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01c6b8-57bb-4776-8fc4-098688b08fcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:40.109413 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01c6b8-57bb-4776-8fc4-098688b08fcb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7b01c6b8-57bb-4776-8fc4-098688b08fcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:43:40.123473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:40.124598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:40.125792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:43:40.125792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:43:40.125792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:43:40.125792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:43:40.125792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:43:40.125792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:43:40.125792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:43:40.125792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:40.125792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 645/1271] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:43:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:40.131015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6dc82-2ef0-49db-b9de-bfc5443e6398 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:40.190180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:40.233073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6dc82-2ef0-49db-b9de-bfc5443e6398 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:40.272400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e30ff-1b9b-4078-bf68-3a5e8b2b746d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:40.273070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e30ff-1b9b-4078-bf68-3a5e8b2b746d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:40.273353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e30ff-1b9b-4078-bf68-3a5e8b2b746d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:40.273629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e30ff-1b9b-4078-bf68-3a5e8b2b746d None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:40.274410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e30ff-1b9b-4078-bf68-3a5e8b2b746d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:40.274721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e30ff-1b9b-4078-bf68-3a5e8b2b746d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:40.276177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e30ff-1b9b-4078-bf68-3a5e8b2b746d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:40.289739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 627/1272] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:40.295897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01c6b8-57bb-4776-8fc4-098688b08fcb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:40.390591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:40.399775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01c6b8-57bb-4776-8fc4-098688b08fcb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:40.434586 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63ab4a-f80e-4e1b-8ffa-a78c0b1df4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:40.435502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63ab4a-f80e-4e1b-8ffa-a78c0b1df4d5 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:40.435781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63ab4a-f80e-4e1b-8ffa-a78c0b1df4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:40.436055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63ab4a-f80e-4e1b-8ffa-a78c0b1df4d5 None admin] PATH_INFO: `/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/7847d2c119474d49bef6029d51dbefdb/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:40.438038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63ab4a-f80e-4e1b-8ffa-a78c0b1df4d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61dea65b34b47a186151c6a63bc7d79, user_id=7847d2c119474d49bef6029d51dbefdb, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:40.439495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63ab4a-f80e-4e1b-8ffa-a78c0b1df4d5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:40.510450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 646/1273] 10.222.0.121 () {58 vars in 1444 bytes} [Thu May 26 17:43:40 2022] PUT /identity/v3/projects/d61dea65b34b47a186151c6a63bc7d79/users/7847d2c119474d49bef6029d51dbefdb/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:43:42.752649 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8187b65-538b-4be2-9f8b-bf56ecee0d61 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:42.753183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8187b65-538b-4be2-9f8b-bf56ecee0d61 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:42.753183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8187b65-538b-4be2-9f8b-bf56ecee0d61 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:42.754576 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 628/1274] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:43:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:43:42.760323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c1316-ca8c-4fdb-8227-0b41ed5d955a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:42.760768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c1316-ca8c-4fdb-8227-0b41ed5d955a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:42.760849 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c1316-ca8c-4fdb-8227-0b41ed5d955a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:42.778351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-303c1316-ca8c-4fdb-8227-0b41ed5d955a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:43:42.783695 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-303c1316-ca8c-4fdb-8227-0b41ed5d955a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:43:42.847433 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 647/1275] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:43:42 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:43:42.872818 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8187b65-538b-4be2-9f8b-bf56ecee0d61 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:42.982391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8187b65-538b-4be2-9f8b-bf56ecee0d61 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:43.001999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:43.018765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4fec2-7d24-47f5-89e2-1e96e9dff22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:43.019536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4fec2-7d24-47f5-89e2-1e96e9dff22c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:43.019913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4fec2-7d24-47f5-89e2-1e96e9dff22c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:43.020156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4fec2-7d24-47f5-89e2-1e96e9dff22c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:43.030513 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4fec2-7d24-47f5-89e2-1e96e9dff22c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-bed4fec2-7d24-47f5-89e2-1e96e9dff22c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:43:43.038242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:43.039387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:43.040538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 629/1276] 10.222.0.121 () {56 vars in 1101 bytes} [Thu May 26 17:43:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:43.046046 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c1316-ca8c-4fdb-8227-0b41ed5d955a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:43.133099 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:43.151735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c1316-ca8c-4fdb-8227-0b41ed5d955a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:43.186476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90339d-dfee-494e-b897-0f98afe6d4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:43.187213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90339d-dfee-494e-b897-0f98afe6d4ae None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:43.187501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90339d-dfee-494e-b897-0f98afe6d4ae None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:43.187778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90339d-dfee-494e-b897-0f98afe6d4ae None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:43.188524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90339d-dfee-494e-b897-0f98afe6d4ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:43.188832 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90339d-dfee-494e-b897-0f98afe6d4ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:43.190211 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90339d-dfee-494e-b897-0f98afe6d4ae None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:43.198370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 648/1277] 10.222.0.121 () {56 vars in 1104 bytes} [Thu May 26 17:43:43 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:43.204769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4fec2-7d24-47f5-89e2-1e96e9dff22c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:43.311297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4fec2-7d24-47f5-89e2-1e96e9dff22c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:43.345831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678160c0-32c3-4b5f-986c-7701f67c9514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:43.346616 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678160c0-32c3-4b5f-986c-7701f67c9514 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:43.346917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678160c0-32c3-4b5f-986c-7701f67c9514 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:43.347185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678160c0-32c3-4b5f-986c-7701f67c9514 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:43.357436 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678160c0-32c3-4b5f-986c-7701f67c9514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:43.358917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678160c0-32c3-4b5f-986c-7701f67c9514 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-678160c0-32c3-4b5f-986c-7701f67c9514 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:43.376116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:43.377194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:43.378354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:43.378354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:43.378354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:43.378354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:43.378354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 630/1278] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:43.383532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90339d-dfee-494e-b897-0f98afe6d4ae None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:43.483851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90339d-dfee-494e-b897-0f98afe6d4ae None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:43.517507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b31a1-fd78-41cb-8f2d-08c1b4921cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:43.518171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b31a1-fd78-41cb-8f2d-08c1b4921cd7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:43.518554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b31a1-fd78-41cb-8f2d-08c1b4921cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:43.518725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b31a1-fd78-41cb-8f2d-08c1b4921cd7 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:43.519468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b31a1-fd78-41cb-8f2d-08c1b4921cd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:43.519767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b31a1-fd78-41cb-8f2d-08c1b4921cd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:43.521130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b31a1-fd78-41cb-8f2d-08c1b4921cd7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:43.533495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 649/1279] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:43.539551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678160c0-32c3-4b5f-986c-7701f67c9514 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:43.570179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:43.638200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678160c0-32c3-4b5f-986c-7701f67c9514 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:43.673093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1568d1b-440d-4a50-8c25-76103acb1d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:43.673906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1568d1b-440d-4a50-8c25-76103acb1d4c None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:43.674193 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1568d1b-440d-4a50-8c25-76103acb1d4c None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:43.674466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1568d1b-440d-4a50-8c25-76103acb1d4c None admin] PATH_INFO: `/v3/users/placement` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:43.681797 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1568d1b-440d-4a50-8c25-76103acb1d4c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:43.683262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1568d1b-440d-4a50-8c25-76103acb1d4c None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-d1568d1b-440d-4a50-8c25-76103acb1d4c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:43:43.698339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:43.699503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:43.705026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:43.705026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:43.705026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:43:43.705026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:43:43.705026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:43:43.705026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:43:43.705026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:43:43.705026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:43:43.705026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 17:43:43.705026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:43.705026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 631/1280] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:43.706121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b31a1-fd78-41cb-8f2d-08c1b4921cd7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:43.710100 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:43.809810 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b31a1-fd78-41cb-8f2d-08c1b4921cd7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:43.846452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a2d15-9008-48bc-9926-d5c36e772608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:43.847411 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a2d15-9008-48bc-9926-d5c36e772608 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:43.847691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a2d15-9008-48bc-9926-d5c36e772608 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:43.847962 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a2d15-9008-48bc-9926-d5c36e772608 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:43.848786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a2d15-9008-48bc-9926-d5c36e772608 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:43.849091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a2d15-9008-48bc-9926-d5c36e772608 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:43.850537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a2d15-9008-48bc-9926-d5c36e772608 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:43.869540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 650/1281] 10.222.0.121 () {56 vars in 1148 bytes} [Thu May 26 17:43:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:43.875546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1568d1b-440d-4a50-8c25-76103acb1d4c None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:43.967991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:43.977251 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1568d1b-440d-4a50-8c25-76103acb1d4c None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:44.010936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266e826-b2f8-4ebc-b282-eeba599d070e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:44.011586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266e826-b2f8-4ebc-b282-eeba599d070e None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:44.011910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266e826-b2f8-4ebc-b282-eeba599d070e None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:44.012182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266e826-b2f8-4ebc-b282-eeba599d070e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:44.079236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266e826-b2f8-4ebc-b282-eeba599d070e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:44.080678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266e826-b2f8-4ebc-b282-eeba599d070e None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:44.183873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:44.194272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 632/1282] 10.222.0.121 () {58 vars in 1287 bytes} [Thu May 26 17:43:43 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 17:43:44.200179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a2d15-9008-48bc-9926-d5c36e772608 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:44.264737 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:44.309734 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a2d15-9008-48bc-9926-d5c36e772608 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:44.347789 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4de27f-208d-41cc-b29c-aad861545a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:44.348626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4de27f-208d-41cc-b29c-aad861545a4c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:44.348907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4de27f-208d-41cc-b29c-aad861545a4c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:44.349178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4de27f-208d-41cc-b29c-aad861545a4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:44.363552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4de27f-208d-41cc-b29c-aad861545a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:44.365048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4de27f-208d-41cc-b29c-aad861545a4c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-be4de27f-208d-41cc-b29c-aad861545a4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:43:44.385397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:44.386525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:44.387477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:43:44.387477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:43:44.387477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:43:44.387477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:44.387477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 651/1283] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:44.393697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266e826-b2f8-4ebc-b282-eeba599d070e None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:44.507674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266e826-b2f8-4ebc-b282-eeba599d070e None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:44.545914 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffc42e-687f-4b16-b7ac-6be2982ba930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:44.546728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffc42e-687f-4b16-b7ac-6be2982ba930 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:44.547040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffc42e-687f-4b16-b7ac-6be2982ba930 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:44.547318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffc42e-687f-4b16-b7ac-6be2982ba930 None admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:44.548065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ffc42e-687f-4b16-b7ac-6be2982ba930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:44.548455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ffc42e-687f-4b16-b7ac-6be2982ba930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:44.549891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ffc42e-687f-4b16-b7ac-6be2982ba930 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:44.568118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 633/1284] 10.222.0.121 () {56 vars in 1114 bytes} [Thu May 26 17:43:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:44.569339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4de27f-208d-41cc-b29c-aad861545a4c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:44.684725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4de27f-208d-41cc-b29c-aad861545a4c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:44.700371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:44.721709 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbaca7-3f88-4a63-82c5-d9fc87556dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:44.721943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbaca7-3f88-4a63-82c5-d9fc87556dca None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:44.722238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbaca7-3f88-4a63-82c5-d9fc87556dca None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:44.722515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbaca7-3f88-4a63-82c5-d9fc87556dca None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:44.729438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbaca7-3f88-4a63-82c5-d9fc87556dca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:44.730930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbaca7-3f88-4a63-82c5-d9fc87556dca None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-d8bbaca7-3f88-4a63-82c5-d9fc87556dca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:43:44.746347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:44.747435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:44.748972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:43:44.748972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:43:44.748972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:43:44.748972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:43:44.748972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:43:44.748972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:43:44.748972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:43:44.748972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application May 26 17:43:44.748972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 652/1285] 10.222.0.121 () {56 vars in 1116 bytes} [Thu May 26 17:43:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:44.754293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffc42e-687f-4b16-b7ac-6be2982ba930 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:44.810558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:44.863711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffc42e-687f-4b16-b7ac-6be2982ba930 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:44.896640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667de5f-8034-4cfb-9450-bf0bcc26c1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:44.897383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667de5f-8034-4cfb-9450-bf0bcc26c1da None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:44.897660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667de5f-8034-4cfb-9450-bf0bcc26c1da None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:44.897925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667de5f-8034-4cfb-9450-bf0bcc26c1da None admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:44.898694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667de5f-8034-4cfb-9450-bf0bcc26c1da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:44.899027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667de5f-8034-4cfb-9450-bf0bcc26c1da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:44.900431 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667de5f-8034-4cfb-9450-bf0bcc26c1da None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:44.912703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 634/1286] 10.222.0.121 () {56 vars in 1153 bytes} [Thu May 26 17:43:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:44.919069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbaca7-3f88-4a63-82c5-d9fc87556dca None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:45.020573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbaca7-3f88-4a63-82c5-d9fc87556dca None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:45.055714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570793d2-0437-4b13-9265-e812facba3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:45.056370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570793d2-0437-4b13-9265-e812facba3ed None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:45.056659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570793d2-0437-4b13-9265-e812facba3ed None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:45.058059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570793d2-0437-4b13-9265-e812facba3ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:45.058059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570793d2-0437-4b13-9265-e812facba3ed None admin] RBAC: Adding query filter params (role.id=4f574961798246a4b12ad2276e67685d, scope.project.id=d61dea65b34b47a186151c6a63bc7d79, user.id=7847d2c119474d49bef6029d51dbefdb) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:45.058643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570793d2-0437-4b13-9265-e812facba3ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:45.059562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570793d2-0437-4b13-9265-e812facba3ed None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:45.065598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 653/1287] 10.222.0.121 () {56 vars in 1379 bytes} [Thu May 26 17:43:44 2022] GET /identity/v3/role_assignments?user.id=7847d2c119474d49bef6029d51dbefdb&scope.project.id=d61dea65b34b47a186151c6a63bc7d79&role.id=4f574961798246a4b12ad2276e67685d => generated 610 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:47.347184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f4017-0227-4625-a221-23d4f0c7ba6b None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:47.347756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f4017-0227-4625-a221-23d4f0c7ba6b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:47.347756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f4017-0227-4625-a221-23d4f0c7ba6b None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:47.349144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 635/1288] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:43:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:43:47.354415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b057c-61c1-480b-9312-67184f2bd038 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:47.354863 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b057c-61c1-480b-9312-67184f2bd038 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:47.355014 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b057c-61c1-480b-9312-67184f2bd038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:47.371879 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f34b057c-61c1-480b-9312-67184f2bd038 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:43:47.373837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f34b057c-61c1-480b-9312-67184f2bd038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:43:47.404837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:47.441776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 654/1289] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:43:47 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:43:47.464666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f4017-0227-4625-a221-23d4f0c7ba6b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:47.568746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f4017-0227-4625-a221-23d4f0c7ba6b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:47.579687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:47.603265 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f502e4c-8b45-4c9b-93b9-c69104cbfd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:47.604068 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f502e4c-8b45-4c9b-93b9-c69104cbfd1d None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:47.604351 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f502e4c-8b45-4c9b-93b9-c69104cbfd1d None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:47.604626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f502e4c-8b45-4c9b-93b9-c69104cbfd1d None admin] PATH_INFO: `/v3/services/placement` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:47.605276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f502e4c-8b45-4c9b-93b9-c69104cbfd1d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:47.606632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f502e4c-8b45-4c9b-93b9-c69104cbfd1d None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-3f502e4c-8b45-4c9b-93b9-c69104cbfd1d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:47.618317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:47.619552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:43:47.620445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:47.623523 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 636/1290] 10.222.0.121 () {56 vars in 1122 bytes} [Thu May 26 17:43:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:47.626105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b057c-61c1-480b-9312-67184f2bd038 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:47.725061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b057c-61c1-480b-9312-67184f2bd038 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:47.757748 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e40086-4702-4d0e-813d-16391cde3cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:47.758407 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e40086-4702-4d0e-813d-16391cde3cb4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:47.758688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e40086-4702-4d0e-813d-16391cde3cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:47.758995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e40086-4702-4d0e-813d-16391cde3cb4 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:47.759700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e40086-4702-4d0e-813d-16391cde3cb4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:47.760006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e40086-4702-4d0e-813d-16391cde3cb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:47.761374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e40086-4702-4d0e-813d-16391cde3cb4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:47.766864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 655/1291] 10.222.0.121 () {56 vars in 1121 bytes} [Thu May 26 17:43:47 2022] GET /identity/v3/services?name=placement => generated 127 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:47.773512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f502e4c-8b45-4c9b-93b9-c69104cbfd1d None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:47.877125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f502e4c-8b45-4c9b-93b9-c69104cbfd1d None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:47.910494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee4bad-9552-40a3-8dbf-7c2f9ed111e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:47.911181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee4bad-9552-40a3-8dbf-7c2f9ed111e7 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:47.911462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee4bad-9552-40a3-8dbf-7c2f9ed111e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:47.911735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee4bad-9552-40a3-8dbf-7c2f9ed111e7 None admin] PATH_INFO: `/v3/services` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:47.912688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee4bad-9552-40a3-8dbf-7c2f9ed111e7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:47.912993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee4bad-9552-40a3-8dbf-7c2f9ed111e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:47.914365 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee4bad-9552-40a3-8dbf-7c2f9ed111e7 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:47.918700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 637/1292] 10.222.0.121 () {56 vars in 1121 bytes} [Thu May 26 17:43:47 2022] GET /identity/v3/services?type=placement => generated 127 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:50.156311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707a894-011f-40f8-80ad-4499434e0284 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:50.156850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707a894-011f-40f8-80ad-4499434e0284 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:50.156850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707a894-011f-40f8-80ad-4499434e0284 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:50.158247 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 656/1293] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:43:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:43:50.163735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e480e9b-2ea1-4842-bcbe-7258569106d8 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:50.164147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e480e9b-2ea1-4842-bcbe-7258569106d8 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:50.164315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e480e9b-2ea1-4842-bcbe-7258569106d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:50.181681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3e480e9b-2ea1-4842-bcbe-7258569106d8 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:43:50.182976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3e480e9b-2ea1-4842-bcbe-7258569106d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:43:50.212565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:50.249881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 638/1294] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:43:50 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:43:50.272820 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707a894-011f-40f8-80ad-4499434e0284 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:50.305443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:50.383226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707a894-011f-40f8-80ad-4499434e0284 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:50.418525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf4bc8-e3f3-436a-a8d5-1d320bb277c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:50.419219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf4bc8-e3f3-436a-a8d5-1d320bb277c4 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:50.419496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf4bc8-e3f3-436a-a8d5-1d320bb277c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:50.420679 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf4bc8-e3f3-436a-a8d5-1d320bb277c4 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:50.421234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf4bc8-e3f3-436a-a8d5-1d320bb277c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:50.422649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf4bc8-e3f3-436a-a8d5-1d320bb277c4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:50.487702 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 657/1295] 10.222.0.121 () {60 vars in 1146 bytes} [Thu May 26 17:43:50 2022] POST /identity/v3/services => generated 247 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:43:52.716199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66f7a1-9c98-4ca6-a153-25c501faaee6 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:52.716732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66f7a1-9c98-4ca6-a153-25c501faaee6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:52.716732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66f7a1-9c98-4ca6-a153-25c501faaee6 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:52.719012 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 639/1296] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:43:52 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:43:52.725374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221840cf-02a2-4591-a235-38a3a1806f5e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:52.725852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221840cf-02a2-4591-a235-38a3a1806f5e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:52.726339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221840cf-02a2-4591-a235-38a3a1806f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:52.746807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-221840cf-02a2-4591-a235-38a3a1806f5e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:43:52.748025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-221840cf-02a2-4591-a235-38a3a1806f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:43:52.821329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 658/1297] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:43:52 2022] POST /identity/v3/auth/tokens => generated 2216 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:43:52.844937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66f7a1-9c98-4ca6-a153-25c501faaee6 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:52.949602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66f7a1-9c98-4ca6-a153-25c501faaee6 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:52.985343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837745a-0148-4ece-8c21-d4503318b0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:52.986153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837745a-0148-4ece-8c21-d4503318b0a5 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:52.986439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837745a-0148-4ece-8c21-d4503318b0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:52.986715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837745a-0148-4ece-8c21-d4503318b0a5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:52.987412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837745a-0148-4ece-8c21-d4503318b0a5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:52.988771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837745a-0148-4ece-8c21-d4503318b0a5 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-8837745a-0148-4ece-8c21-d4503318b0a5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:53.000813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:53.007146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:53.008285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 640/1298] 10.222.0.121 () {56 vars in 1122 bytes} [Thu May 26 17:43:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:53.009085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221840cf-02a2-4591-a235-38a3a1806f5e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:53.027453 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:53.109596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221840cf-02a2-4591-a235-38a3a1806f5e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:53.143884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcea618-4495-44d6-ae91-6212b2c5f7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:53.144558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcea618-4495-44d6-ae91-6212b2c5f7a7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:53.144840 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcea618-4495-44d6-ae91-6212b2c5f7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:53.145113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcea618-4495-44d6-ae91-6212b2c5f7a7 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:53.145813 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcea618-4495-44d6-ae91-6212b2c5f7a7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:53.146117 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcea618-4495-44d6-ae91-6212b2c5f7a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:53.147517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcea618-4495-44d6-ae91-6212b2c5f7a7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:53.152643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 659/1299] 10.222.0.121 () {56 vars in 1121 bytes} [Thu May 26 17:43:53 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:43:53.159301 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837745a-0148-4ece-8c21-d4503318b0a5 None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:53.195311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:53.266182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837745a-0148-4ece-8c21-d4503318b0a5 None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:53.301276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fb26c-dc5c-4924-b56f-e289e07ff719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:53.301934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fb26c-dc5c-4924-b56f-e289e07ff719 None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:53.302215 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fb26c-dc5c-4924-b56f-e289e07ff719 None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:53.302490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fb26c-dc5c-4924-b56f-e289e07ff719 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:53.303318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fb26c-dc5c-4924-b56f-e289e07ff719 None admin] RBAC: Adding query filter params (interface=public, service_id=7eb2017ba67e4475a6f96a24d85652a9, region_id=RegionOne) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:53.303634 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fb26c-dc5c-4924-b56f-e289e07ff719 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:53.305115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fb26c-dc5c-4924-b56f-e289e07ff719 None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:53.310385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 641/1300] 10.222.0.121 () {56 vars in 1256 bytes} [Thu May 26 17:43:53 2022] GET /identity/v3/endpoints?service_id=7eb2017ba67e4475a6f96a24d85652a9&interface=public®ion_id=RegionOne => generated 195 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:53.316862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcea618-4495-44d6-ae91-6212b2c5f7a7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:53.399910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:53.417308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcea618-4495-44d6-ae91-6212b2c5f7a7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:53.452587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac41ec-26c3-44c1-9a6a-4a1619e67722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:53.453305 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac41ec-26c3-44c1-9a6a-4a1619e67722 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:53.454010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac41ec-26c3-44c1-9a6a-4a1619e67722 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:53.454321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac41ec-26c3-44c1-9a6a-4a1619e67722 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:53.455489 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac41ec-26c3-44c1-9a6a-4a1619e67722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:53.456889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac41ec-26c3-44c1-9a6a-4a1619e67722 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:53.463920 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 660/1301] 10.222.0.121 () {56 vars in 1092 bytes} [Thu May 26 17:43:53 2022] GET /identity/v3/services => generated 2201 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:43:55.659176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128c0b9-ccaa-4b52-b223-c42fe3952f18 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:55.660032 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128c0b9-ccaa-4b52-b223-c42fe3952f18 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:55.660032 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128c0b9-ccaa-4b52-b223-c42fe3952f18 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:55.661560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 642/1302] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:43:55 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:43:55.667122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c7808-c680-466e-a23b-6eeb32db57ca None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:55.667552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c7808-c680-466e-a23b-6eeb32db57ca None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:55.667682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c7808-c680-466e-a23b-6eeb32db57ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:55.684400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-319c7808-c680-466e-a23b-6eeb32db57ca None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:43:55.686250 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-319c7808-c680-466e-a23b-6eeb32db57ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:43:55.752137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 661/1303] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:43:55 2022] POST /identity/v3/auth/tokens => generated 2216 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:43:55.775188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128c0b9-ccaa-4b52-b223-c42fe3952f18 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:55.875243 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:55.883993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128c0b9-ccaa-4b52-b223-c42fe3952f18 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:55.918423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2827086-aa47-49ef-9c94-698173c5cdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:55.919173 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2827086-aa47-49ef-9c94-698173c5cdcb None admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:55.919453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2827086-aa47-49ef-9c94-698173c5cdcb None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:55.919723 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2827086-aa47-49ef-9c94-698173c5cdcb None admin] PATH_INFO: `/v3/services/placement` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:55.921359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2827086-aa47-49ef-9c94-698173c5cdcb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:55.922719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2827086-aa47-49ef-9c94-698173c5cdcb None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application [None req-e2827086-aa47-49ef-9c94-698173c5cdcb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:43:55.934642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:43:55.936437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application with Lock( May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self._enter() May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application return self.creator() May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application created_value = creator( May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: ERROR keystone.server.flask.application May 26 17:43:55.937427 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 643/1304] 10.222.0.121 () {56 vars in 1122 bytes} [Thu May 26 17:43:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:43:55.943168 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c7808-c680-466e-a23b-6eeb32db57ca None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:56.020065 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:43:56.043727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c7808-c680-466e-a23b-6eeb32db57ca None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:56.082796 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ad019-25ef-4a58-b738-9a1b19c5b983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:56.083486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ad019-25ef-4a58-b738-9a1b19c5b983 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:56.083770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ad019-25ef-4a58-b738-9a1b19c5b983 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:56.084048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ad019-25ef-4a58-b738-9a1b19c5b983 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:56.084749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ad019-25ef-4a58-b738-9a1b19c5b983 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:43:56.085055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ad019-25ef-4a58-b738-9a1b19c5b983 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:56.086430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ad019-25ef-4a58-b738-9a1b19c5b983 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:56.091887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 662/1305] 10.222.0.121 () {56 vars in 1121 bytes} [Thu May 26 17:43:55 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:43:56.097696 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2827086-aa47-49ef-9c94-698173c5cdcb None admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:43:56.198923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2827086-aa47-49ef-9c94-698173c5cdcb None admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:43:56.231903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c0f08-d4dc-46bd-a6f8-fc139f5fcf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:43:56.233152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c0f08-d4dc-46bd-a6f8-fc139f5fcf6b None admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:43:56.233499 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c0f08-d4dc-46bd-a6f8-fc139f5fcf6b None admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:43:56.233834 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c0f08-d4dc-46bd-a6f8-fc139f5fcf6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:43:56.234555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c0f08-d4dc-46bd-a6f8-fc139f5fcf6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:43:56.235936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c0f08-d4dc-46bd-a6f8-fc139f5fcf6b None admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:43:56.321306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 644/1306] 10.222.0.121 () {60 vars in 1149 bytes} [Thu May 26 17:43:56 2022] POST /identity/v3/endpoints => generated 333 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:44:06.088553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=30 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:44:06.316503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:12.853545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870dfade-2c16-401f-933a-62ec96543917 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:12.854116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870dfade-2c16-401f-933a-62ec96543917 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:12.854116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870dfade-2c16-401f-933a-62ec96543917 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:12.861587 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2899ca-75d6-4605-8fb6-6ef064cf8bc1 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:12.864134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2899ca-75d6-4605-8fb6-6ef064cf8bc1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:12.864134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2899ca-75d6-4605-8fb6-6ef064cf8bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:12.865511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 663/1307] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:12 2022] GET /identity => generated 271 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:50:12.960360 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-ae2899ca-75d6-4605-8fb6-6ef064cf8bc1 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:12.961500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ae2899ca-75d6-4605-8fb6-6ef064cf8bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:13.092675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 645/1308] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:13.135412 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ab3eb-9a01-402c-bc8b-187e567a4900 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:13.135769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ab3eb-9a01-402c-bc8b-187e567a4900 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:13.135962 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ab3eb-9a01-402c-bc8b-187e567a4900 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:13.137700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 664/1309] 10.222.0.121 () {54 vars in 969 bytes} [Thu May 26 17:50:13 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:50:13.149091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2631e-6c93-4529-ad0f-59eaed36c31c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:13.149574 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2631e-6c93-4529-ad0f-59eaed36c31c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:13.150067 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2631e-6c93-4529-ad0f-59eaed36c31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:13.317769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-32e2631e-6c93-4529-ad0f-59eaed36c31c None None] MFA Rules not processed for user `08709b462300437598c8db331e5ad20f`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:13.319787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-32e2631e-6c93-4529-ad0f-59eaed36c31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:13.350380 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:13.427018 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 646/1310] 10.222.0.121 () {58 vars in 1066 bytes} [Thu May 26 17:50:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:13.433376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ab3eb-9a01-402c-bc8b-187e567a4900 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:13.435238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-765ab3eb-9a01-402c-bc8b-187e567a4900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:13.565436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ab3eb-9a01-402c-bc8b-187e567a4900 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:13.647262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b01824-9c4e-44c6-8c32-3b0b0d07519c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:13.648057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b01824-9c4e-44c6-8c32-3b0b0d07519c service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:13.648336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b01824-9c4e-44c6-8c32-3b0b0d07519c service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:13.648607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b01824-9c4e-44c6-8c32-3b0b0d07519c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:13.650350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f3b01824-9c4e-44c6-8c32-3b0b0d07519c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:13.714579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:13.777322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b01824-9c4e-44c6-8c32-3b0b0d07519c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:13.778771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b01824-9c4e-44c6-8c32-3b0b0d07519c service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:13.800520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 665/1311] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:50:16.336570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071c5d-b478-4741-b369-881a1c3c1b70 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:16.336570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071c5d-b478-4741-b369-881a1c3c1b70 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:16.336570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071c5d-b478-4741-b369-881a1c3c1b70 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:16.336570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 647/1312] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:50:16.343365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9be74-f5fa-48db-bd94-f73ed4f6f72d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:16.343857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9be74-f5fa-48db-bd94-f73ed4f6f72d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:16.347683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9be74-f5fa-48db-bd94-f73ed4f6f72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:16.432556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9ce9be74-f5fa-48db-bd94-f73ed4f6f72d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:16.433748 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9ce9be74-f5fa-48db-bd94-f73ed4f6f72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:16.539206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 666/1313] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:16.591923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c1c65-4c94-4283-94dc-ad97751aa17c None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:16.592530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c1c65-4c94-4283-94dc-ad97751aa17c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:16.593726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c1c65-4c94-4283-94dc-ad97751aa17c None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:16.594351 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 648/1314] 10.222.0.121 () {54 vars in 969 bytes} [Thu May 26 17:50:16 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:50:16.600627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a9f9b-2251-40df-859e-792fba240c92 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:16.600918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a9f9b-2251-40df-859e-792fba240c92 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:16.601195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a9f9b-2251-40df-859e-792fba240c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:16.612455 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:16.681022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-223a9f9b-2251-40df-859e-792fba240c92 None None] MFA Rules not processed for user `08709b462300437598c8db331e5ad20f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:16.682142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-223a9f9b-2251-40df-859e-792fba240c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:16.764128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 667/1315] 10.222.0.121 () {58 vars in 1066 bytes} [Thu May 26 17:50:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:16.773370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c1c65-4c94-4283-94dc-ad97751aa17c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:16.775369 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7c7c1c65-4c94-4283-94dc-ad97751aa17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:16.884377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c1c65-4c94-4283-94dc-ad97751aa17c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:16.957418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:16.959713 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5666e6-fd1e-47d8-a2f0-2274bda48d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:16.960361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5666e6-fd1e-47d8-a2f0-2274bda48d86 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:16.960640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5666e6-fd1e-47d8-a2f0-2274bda48d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:16.960910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5666e6-fd1e-47d8-a2f0-2274bda48d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:16.962551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4b5666e6-fd1e-47d8-a2f0-2274bda48d86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:17.077616 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5666e6-fd1e-47d8-a2f0-2274bda48d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:17.078959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5666e6-fd1e-47d8-a2f0-2274bda48d86 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:17.101905 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 649/1316] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:50:20.207616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bfaa7-e25a-4e7d-a938-539ce41c2ff6 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:20.208150 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bfaa7-e25a-4e7d-a938-539ce41c2ff6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:20.208244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bfaa7-e25a-4e7d-a938-539ce41c2ff6 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:20.209603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 668/1317] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:50:20.214971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c3e1a-c223-4e24-99a8-00fea3c45b1c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:20.215382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c3e1a-c223-4e24-99a8-00fea3c45b1c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:20.215531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c3e1a-c223-4e24-99a8-00fea3c45b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:20.284854 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a24c3e1a-c223-4e24-99a8-00fea3c45b1c None None] MFA Rules not processed for user `7201ba55061e4bf4882217b64adb67b6`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:20.286069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a24c3e1a-c223-4e24-99a8-00fea3c45b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:20.341108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:20.388864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 650/1318] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:20.429675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bfaa7-e25a-4e7d-a938-539ce41c2ff6 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:20.431456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-da3bfaa7-e25a-4e7d-a938-539ce41c2ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:20.536942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bfaa7-e25a-4e7d-a938-539ce41c2ff6 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:20.539855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:20.609845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39254b-b0d5-4979-978d-436c82d76f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:20.610521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39254b-b0d5-4979-978d-436c82d76f7c service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:20.610796 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39254b-b0d5-4979-978d-436c82d76f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:20.611099 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39254b-b0d5-4979-978d-436c82d76f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:20.612844 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2b39254b-b0d5-4979-978d-436c82d76f7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:20.730650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39254b-b0d5-4979-978d-436c82d76f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:20.732679 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39254b-b0d5-4979-978d-436c82d76f7c service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:20.751620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 669/1319] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:50:24.038029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd5ee3-4bf7-4ee9-97b8-c92bb812dd66 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:24.038626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd5ee3-4bf7-4ee9-97b8-c92bb812dd66 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:24.038626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd5ee3-4bf7-4ee9-97b8-c92bb812dd66 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:24.040016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 651/1320] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:50:24.045699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99d7bf-478d-4b42-a045-a9a46ba99622 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:24.046138 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99d7bf-478d-4b42-a045-a9a46ba99622 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:24.046250 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99d7bf-478d-4b42-a045-a9a46ba99622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:24.085362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:24.111798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-cb99d7bf-478d-4b42-a045-a9a46ba99622 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:24.113003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cb99d7bf-478d-4b42-a045-a9a46ba99622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:24.219753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 670/1321] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:24.257088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd5ee3-4bf7-4ee9-97b8-c92bb812dd66 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:24.258705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e4cd5ee3-4bf7-4ee9-97b8-c92bb812dd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:24.365332 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd5ee3-4bf7-4ee9-97b8-c92bb812dd66 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:24.443990 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c924e3b-207b-4a62-aacd-394c916c3d03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:24.444638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c924e3b-207b-4a62-aacd-394c916c3d03 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:24.444918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c924e3b-207b-4a62-aacd-394c916c3d03 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:24.445188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c924e3b-207b-4a62-aacd-394c916c3d03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:24.446865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-8c924e3b-207b-4a62-aacd-394c916c3d03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:24.454501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:24.558020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c924e3b-207b-4a62-aacd-394c916c3d03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:24.559485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c924e3b-207b-4a62-aacd-394c916c3d03 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:24.579783 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 652/1322] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:50:27.964846 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb6da-78b4-4603-8a8b-4947cf615fd7 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:27.965758 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb6da-78b4-4603-8a8b-4947cf615fd7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:27.965758 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb6da-78b4-4603-8a8b-4947cf615fd7 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:27.967774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 671/1323] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:27 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:50:27.974452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99fbe3-e78c-41a0-a46e-694f9c430f9a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:27.975302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99fbe3-e78c-41a0-a46e-694f9c430f9a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:27.975560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99fbe3-e78c-41a0-a46e-694f9c430f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:28.078734 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-fd99fbe3-e78c-41a0-a46e-694f9c430f9a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:28.080572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-fd99fbe3-e78c-41a0-a46e-694f9c430f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:28.182554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:28.238288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 653/1324] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:50:28.293559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cb6da-78b4-4603-8a8b-4947cf615fd7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:28.296589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e30cb6da-78b4-4603-8a8b-4947cf615fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:28.458534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cb6da-78b4-4603-8a8b-4947cf615fd7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:28.477653 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:28.555591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c7a43-1bc5-4153-8e67-d7467f207fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:28.556258 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c7a43-1bc5-4153-8e67-d7467f207fdc service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:28.556543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c7a43-1bc5-4153-8e67-d7467f207fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:28.556821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c7a43-1bc5-4153-8e67-d7467f207fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:28.558691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6f7c7a43-1bc5-4153-8e67-d7467f207fdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:28.678460 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c7a43-1bc5-4153-8e67-d7467f207fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:28.679936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c7a43-1bc5-4153-8e67-d7467f207fdc service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:28.699568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 672/1325] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:50:31.036666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8b1e5-d171-456d-affc-48d734316296 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:31.037232 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8b1e5-d171-456d-affc-48d734316296 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:31.037232 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8b1e5-d171-456d-affc-48d734316296 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:31.038666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 654/1326] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:50:31.044549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28795a99-bd01-4a35-8909-81a5beb5945f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:31.045048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28795a99-bd01-4a35-8909-81a5beb5945f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:31.045154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28795a99-bd01-4a35-8909-81a5beb5945f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:31.098063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:31.119203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-28795a99-bd01-4a35-8909-81a5beb5945f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:31.120459 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-28795a99-bd01-4a35-8909-81a5beb5945f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:31.235134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 673/1327] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:31.271781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8b1e5-d171-456d-affc-48d734316296 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:31.273441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d5a8b1e5-d171-456d-affc-48d734316296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:31.395131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8b1e5-d171-456d-affc-48d734316296 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:31.486044 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd771f-e40c-4623-9902-ee0dc638266d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:31.486722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd771f-e40c-4623-9902-ee0dc638266d service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:31.487098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd771f-e40c-4623-9902-ee0dc638266d service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:31.487430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd771f-e40c-4623-9902-ee0dc638266d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:31.489168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5dcd771f-e40c-4623-9902-ee0dc638266d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:31.508754 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:31.616927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd771f-e40c-4623-9902-ee0dc638266d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:31.618387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd771f-e40c-4623-9902-ee0dc638266d service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:31.641459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 655/1328] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:50:34.066347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ead0bb-3f68-4d70-a090-099e44b3d921 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:34.067418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ead0bb-3f68-4d70-a090-099e44b3d921 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:34.067713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ead0bb-3f68-4d70-a090-099e44b3d921 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:34.069273 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 674/1329] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:34 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:50:34.076056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699298ff-ff2a-4c53-9ddc-afbd066145c6 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:34.076882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699298ff-ff2a-4c53-9ddc-afbd066145c6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:34.076882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699298ff-ff2a-4c53-9ddc-afbd066145c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:34.283698 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-699298ff-ff2a-4c53-9ddc-afbd066145c6 None None] MFA Rules not processed for user `7201ba55061e4bf4882217b64adb67b6`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:34.284908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-699298ff-ff2a-4c53-9ddc-afbd066145c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:34.351624 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:34.384364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 656/1330] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:34 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:34.423362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ead0bb-3f68-4d70-a090-099e44b3d921 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:34.424986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a5ead0bb-3f68-4d70-a090-099e44b3d921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:34.532304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ead0bb-3f68-4d70-a090-099e44b3d921 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:34.553699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:34.613171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ce46b-7c40-4637-a582-b61cf34a6ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:34.613877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ce46b-7c40-4637-a582-b61cf34a6ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:34.614159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ce46b-7c40-4637-a582-b61cf34a6ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:34.614430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ce46b-7c40-4637-a582-b61cf34a6ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:34.616089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-131ce46b-7c40-4637-a582-b61cf34a6ed8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:34.740492 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ce46b-7c40-4637-a582-b61cf34a6ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:34.742020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ce46b-7c40-4637-a582-b61cf34a6ed8 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:34.764571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 675/1331] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:34 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 17:50:37.972747 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76884777-86ce-4d18-922a-25e49c0bbcdd None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:37.973324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76884777-86ce-4d18-922a-25e49c0bbcdd None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:37.973324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76884777-86ce-4d18-922a-25e49c0bbcdd None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:37.974664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 657/1332] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:50:37.980308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c59b7-4afb-48b1-90d2-e2c22a08ea70 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:37.980794 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c59b7-4afb-48b1-90d2-e2c22a08ea70 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:37.980907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c59b7-4afb-48b1-90d2-e2c22a08ea70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:38.056964 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:38.060692 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ad5c59b7-4afb-48b1-90d2-e2c22a08ea70 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:38.062024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ad5c59b7-4afb-48b1-90d2-e2c22a08ea70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:38.174290 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 676/1333] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:38.211949 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76884777-86ce-4d18-922a-25e49c0bbcdd None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:38.213583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-76884777-86ce-4d18-922a-25e49c0bbcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:38.330313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76884777-86ce-4d18-922a-25e49c0bbcdd None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:38.422390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1101811-8522-42d4-8069-20752b641d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:38.422390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1101811-8522-42d4-8069-20752b641d41 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:38.422390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1101811-8522-42d4-8069-20752b641d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:38.422390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1101811-8522-42d4-8069-20752b641d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:38.422971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d1101811-8522-42d4-8069-20752b641d41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:38.450955 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:38.536124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1101811-8522-42d4-8069-20752b641d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:38.537583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1101811-8522-42d4-8069-20752b641d41 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:38.557971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 658/1334] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:50:40.935111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba4f76-d09d-41aa-b702-3ecd3fc98439 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:40.935683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba4f76-d09d-41aa-b702-3ecd3fc98439 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:40.935683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba4f76-d09d-41aa-b702-3ecd3fc98439 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:40.937185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 677/1335] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:50:40.944321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13917d85-0ac7-4807-9ec4-c347e0a9cb54 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:40.944321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13917d85-0ac7-4807-9ec4-c347e0a9cb54 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:40.944321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13917d85-0ac7-4807-9ec4-c347e0a9cb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:41.016204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-13917d85-0ac7-4807-9ec4-c347e0a9cb54 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:41.017424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-13917d85-0ac7-4807-9ec4-c347e0a9cb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:41.105932 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:41.132272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 659/1336] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:41.174512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba4f76-d09d-41aa-b702-3ecd3fc98439 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:41.176128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0bba4f76-d09d-41aa-b702-3ecd3fc98439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:41.290495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba4f76-d09d-41aa-b702-3ecd3fc98439 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:41.321079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:41.369783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1816fde-be8c-463e-b2e3-e1bd93c3dd84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:41.370388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1816fde-be8c-463e-b2e3-e1bd93c3dd84 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:41.370666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1816fde-be8c-463e-b2e3-e1bd93c3dd84 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:41.370963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1816fde-be8c-463e-b2e3-e1bd93c3dd84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:41.372715 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f1816fde-be8c-463e-b2e3-e1bd93c3dd84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:41.489131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1816fde-be8c-463e-b2e3-e1bd93c3dd84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:41.490655 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1816fde-be8c-463e-b2e3-e1bd93c3dd84 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:41.509797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 678/1337] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:50:44.736137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66196c42-1610-46f4-92ba-003be5a4350b None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:44.736876 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66196c42-1610-46f4-92ba-003be5a4350b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:44.736876 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66196c42-1610-46f4-92ba-003be5a4350b None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:44.738069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 660/1338] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:44 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:50:44.744672 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bd305-99db-4cbc-934a-737e14088b92 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:44.745182 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bd305-99db-4cbc-934a-737e14088b92 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:44.745612 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bd305-99db-4cbc-934a-737e14088b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:44.832423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2d3bd305-99db-4cbc-934a-737e14088b92 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:44.833640 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2d3bd305-99db-4cbc-934a-737e14088b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:44.842151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:44.954216 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 679/1339] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:44.992092 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66196c42-1610-46f4-92ba-003be5a4350b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:44.994784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-66196c42-1610-46f4-92ba-003be5a4350b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:45.099180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66196c42-1610-46f4-92ba-003be5a4350b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:45.103332 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a4c66-aa5d-4e04-b06f-b98af739701e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:45.103995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a4c66-aa5d-4e04-b06f-b98af739701e service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:45.104273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a4c66-aa5d-4e04-b06f-b98af739701e service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:45.104542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a4c66-aa5d-4e04-b06f-b98af739701e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:45.115025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a4c66-aa5d-4e04-b06f-b98af739701e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:45.116464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a4c66-aa5d-4e04-b06f-b98af739701e service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:45.126674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 661/1340] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:50:49.990537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372ed88-a035-45a4-90a4-7c1cced783b4 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.991283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372ed88-a035-45a4-90a4-7c1cced783b4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.991283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372ed88-a035-45a4-90a4-7c1cced783b4 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.992521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 680/1341] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:50:49.998229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86a90b-2fc6-4fea-89b7-31e19f33bd84 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.999739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86a90b-2fc6-4fea-89b7-31e19f33bd84 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.999739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86a90b-2fc6-4fea-89b7-31e19f33bd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:50.030017 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1c86a90b-2fc6-4fea-89b7-31e19f33bd84 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:50.031504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1c86a90b-2fc6-4fea-89b7-31e19f33bd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:50.139700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 662/1342] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:50.179130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372ed88-a035-45a4-90a4-7c1cced783b4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:50.180972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0372ed88-a035-45a4-90a4-7c1cced783b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:50.194297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372ed88-a035-45a4-90a4-7c1cced783b4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:50.198173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74458bf7-02c7-48c7-8125-3201f2df5a20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:50.198843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74458bf7-02c7-48c7-8125-3201f2df5a20 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:50.199136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74458bf7-02c7-48c7-8125-3201f2df5a20 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:50.199408 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74458bf7-02c7-48c7-8125-3201f2df5a20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:50.210646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74458bf7-02c7-48c7-8125-3201f2df5a20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:50.212139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74458bf7-02c7-48c7-8125-3201f2df5a20 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:50.223417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 681/1343] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:50:53.593825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc0124-25d1-4468-9d21-c0ccf2d30300 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:53.594391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc0124-25d1-4468-9d21-c0ccf2d30300 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:53.594391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc0124-25d1-4468-9d21-c0ccf2d30300 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:53.595691 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 663/1344] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:50:53.601073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cb704-6791-4804-9422-3d696e8a3bf8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:53.601532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cb704-6791-4804-9422-3d696e8a3bf8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:53.601634 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cb704-6791-4804-9422-3d696e8a3bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:53.620247 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8b9cb704-6791-4804-9422-3d696e8a3bf8 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:53.621479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8b9cb704-6791-4804-9422-3d696e8a3bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:53.632336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 682/1345] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:53.670478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc0124-25d1-4468-9d21-c0ccf2d30300 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:53.687335 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc0124-25d1-4468-9d21-c0ccf2d30300 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:53.693162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc5dac-8b48-4caa-9554-8f610a902c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:53.693821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc5dac-8b48-4caa-9554-8f610a902c85 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:53.694102 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc5dac-8b48-4caa-9554-8f610a902c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:53.694376 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc5dac-8b48-4caa-9554-8f610a902c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:53.705832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc5dac-8b48-4caa-9554-8f610a902c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:53.707471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc5dac-8b48-4caa-9554-8f610a902c85 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:53.718280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 664/1346] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:50:58.228824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a57e6-84e4-4560-ab6e-eb654fbd236b None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.229347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a57e6-84e4-4560-ab6e-eb654fbd236b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.229347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a57e6-84e4-4560-ab6e-eb654fbd236b None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.230707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 683/1347] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:50:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:50:58.236189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b603b-5e47-4fc4-96b1-4706b11bfcc3 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.236623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b603b-5e47-4fc4-96b1-4706b11bfcc3 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.236744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b603b-5e47-4fc4-96b1-4706b11bfcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.257481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f67b603b-5e47-4fc4-96b1-4706b11bfcc3 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:58.258695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f67b603b-5e47-4fc4-96b1-4706b11bfcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:58.267441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 665/1348] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:50:58.305786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a57e6-84e4-4560-ab6e-eb654fbd236b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:58.317381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a57e6-84e4-4560-ab6e-eb654fbd236b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:58.320894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3f0a7-9dd1-41b3-ab9a-84e8f5b27bfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:58.321544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3f0a7-9dd1-41b3-ab9a-84e8f5b27bfa service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.321823 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3f0a7-9dd1-41b3-ab9a-84e8f5b27bfa service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.322096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3f0a7-9dd1-41b3-ab9a-84e8f5b27bfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.332065 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3f0a7-9dd1-41b3-ab9a-84e8f5b27bfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:58.333509 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3f0a7-9dd1-41b3-ab9a-84e8f5b27bfa service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:58.344479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 684/1349] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:51:03.952041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de5fc4-7681-4e05-8153-2dad6a994b00 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.952662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de5fc4-7681-4e05-8153-2dad6a994b00 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.952830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de5fc4-7681-4e05-8153-2dad6a994b00 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.954452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 666/1350] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 17:51:03 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:03.960043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bbe58-f47c-46b0-ad16-bd8bd37dfec0 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.960999 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bbe58-f47c-46b0-ad16-bd8bd37dfec0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.961485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bbe58-f47c-46b0-ad16-bd8bd37dfec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:04.040732 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-251bbe58-f47c-46b0-ad16-bd8bd37dfec0 None None] MFA Rules not processed for user `7847d2c119474d49bef6029d51dbefdb`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:04.042504 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-251bbe58-f47c-46b0-ad16-bd8bd37dfec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:04.103411 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:04.137091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 685/1351] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 17:51:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:08.252573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=98 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:08.572833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a331f2-76f0-4ecb-92da-6f40001f1b6f None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:08.573405 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a331f2-76f0-4ecb-92da-6f40001f1b6f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:08.574074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a331f2-76f0-4ecb-92da-6f40001f1b6f None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:08.576146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 667/1352] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 17:51:08 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:08.582728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01d7fd-c1ef-44cb-b91e-b1bafeedbb24 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:08.583323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01d7fd-c1ef-44cb-b91e-b1bafeedbb24 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:08.583944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01d7fd-c1ef-44cb-b91e-b1bafeedbb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:08.605341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-df01d7fd-c1ef-44cb-b91e-b1bafeedbb24 None None] MFA Rules not processed for user `7847d2c119474d49bef6029d51dbefdb`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:08.607184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-df01d7fd-c1ef-44cb-b91e-b1bafeedbb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:08.628550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 686/1353] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 17:51:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:51:10.344035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132503b4-41a8-4db6-8021-98b593b8b437 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:10.344549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132503b4-41a8-4db6-8021-98b593b8b437 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:10.344549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132503b4-41a8-4db6-8021-98b593b8b437 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.345853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 668/1354] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 17:51:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:10.351967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783376a1-0480-4353-ac26-328460934b19 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:10.352401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783376a1-0480-4353-ac26-328460934b19 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:10.352496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783376a1-0480-4353-ac26-328460934b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.371757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-783376a1-0480-4353-ac26-328460934b19 None None] MFA Rules not processed for user `7847d2c119474d49bef6029d51dbefdb`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:10.372925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-783376a1-0480-4353-ac26-328460934b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:10.381172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 687/1355] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:14.261746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416710a-b428-4b7d-998a-559a2126e8de None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.262314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416710a-b428-4b7d-998a-559a2126e8de None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.262314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416710a-b428-4b7d-998a-559a2126e8de None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.263750 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 669/1356] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:14.269368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e89253-215f-4a55-ac95-57d87f6ed4c2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.269815 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e89253-215f-4a55-ac95-57d87f6ed4c2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.269928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e89253-215f-4a55-ac95-57d87f6ed4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.289426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-65e89253-215f-4a55-ac95-57d87f6ed4c2 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:14.290632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-65e89253-215f-4a55-ac95-57d87f6ed4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:14.299877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 688/1357] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:14.355425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31a969-ee59-4b07-b0fc-9d1f97223883 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.355993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31a969-ee59-4b07-b0fc-9d1f97223883 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.356485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31a969-ee59-4b07-b0fc-9d1f97223883 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.358004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 670/1358] 10.222.0.121 () {54 vars in 966 bytes} [Thu May 26 17:51:14 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:14.366805 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3c292-4cc1-487a-9ff6-dac76a6306fd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.367124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3c292-4cc1-487a-9ff6-dac76a6306fd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.367590 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3c292-4cc1-487a-9ff6-dac76a6306fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.434401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-22e3c292-4cc1-487a-9ff6-dac76a6306fd None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:14.435752 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-22e3c292-4cc1-487a-9ff6-dac76a6306fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:14.541710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 689/1359] 10.222.0.121 () {58 vars in 1063 bytes} [Thu May 26 17:51:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:14.550375 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31a969-ee59-4b07-b0fc-9d1f97223883 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:14.568029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31a969-ee59-4b07-b0fc-9d1f97223883 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:14.574612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d55422-1f4d-4355-8054-1dd761787f80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:14.575468 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d55422-1f4d-4355-8054-1dd761787f80 service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.575855 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d55422-1f4d-4355-8054-1dd761787f80 service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.576236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d55422-1f4d-4355-8054-1dd761787f80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.585561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d55422-1f4d-4355-8054-1dd761787f80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:14.587309 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d55422-1f4d-4355-8054-1dd761787f80 service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:14.599143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 671/1360] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:51:17.118259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ff903-0178-4ce2-ae5a-b3cceeedf5b8 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:17.118770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ff903-0178-4ce2-ae5a-b3cceeedf5b8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:17.118888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ff903-0178-4ce2-ae5a-b3cceeedf5b8 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:17.121254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 690/1361] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:17.127285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ad5a4-02dc-4eaf-8da9-284e9bca2b4e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:17.129727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ad5a4-02dc-4eaf-8da9-284e9bca2b4e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:17.129727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ad5a4-02dc-4eaf-8da9-284e9bca2b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:17.146615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-917ad5a4-02dc-4eaf-8da9-284e9bca2b4e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:17.147770 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-917ad5a4-02dc-4eaf-8da9-284e9bca2b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:17.156463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 672/1362] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:51:17.201080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ff903-0178-4ce2-ae5a-b3cceeedf5b8 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:17.216662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ff903-0178-4ce2-ae5a-b3cceeedf5b8 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:17.222233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99866d47-6614-4238-9c29-94f5d30018f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:17.222923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866d47-6614-4238-9c29-94f5d30018f0 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:17.223210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866d47-6614-4238-9c29-94f5d30018f0 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:17.223482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866d47-6614-4238-9c29-94f5d30018f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:17.232586 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99866d47-6614-4238-9c29-94f5d30018f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:17.234206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99866d47-6614-4238-9c29-94f5d30018f0 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:17.246887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 691/1363] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:51:19.989052 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7005478-dba2-4382-9008-33b02f950838 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:19.990010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7005478-dba2-4382-9008-33b02f950838 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:19.991129 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7005478-dba2-4382-9008-33b02f950838 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:19.993620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 673/1364] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:19 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:19.999066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbd088-0040-407a-822f-4f1efe5af72e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:19.999529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbd088-0040-407a-822f-4f1efe5af72e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:19.999642 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbd088-0040-407a-822f-4f1efe5af72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.021636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1bcbd088-0040-407a-822f-4f1efe5af72e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:20.023041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1bcbd088-0040-407a-822f-4f1efe5af72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:20.032876 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 692/1365] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:20.087318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a0f07-e1e6-47f3-a499-e1b1870e298b None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.087671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a0f07-e1e6-47f3-a499-e1b1870e298b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.087978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a0f07-e1e6-47f3-a499-e1b1870e298b None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.089910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 674/1366] 10.222.0.121 () {54 vars in 966 bytes} [Thu May 26 17:51:20 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:20.098843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ae86b-a2c6-40af-90ab-928e79b3ddba None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.098843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ae86b-a2c6-40af-90ab-928e79b3ddba None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.098843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ae86b-a2c6-40af-90ab-928e79b3ddba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.126637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2c7ae86b-a2c6-40af-90ab-928e79b3ddba None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:20.127881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2c7ae86b-a2c6-40af-90ab-928e79b3ddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:20.139081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 693/1367] 10.222.0.121 () {58 vars in 1063 bytes} [Thu May 26 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:20.147479 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a0f07-e1e6-47f3-a499-e1b1870e298b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:20.162652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a0f07-e1e6-47f3-a499-e1b1870e298b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:20.168657 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5563afe-ecbb-4a63-8fd0-5dab0e041858 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:20.169573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5563afe-ecbb-4a63-8fd0-5dab0e041858 service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.169859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5563afe-ecbb-4a63-8fd0-5dab0e041858 service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.170133 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5563afe-ecbb-4a63-8fd0-5dab0e041858 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.178639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5563afe-ecbb-4a63-8fd0-5dab0e041858 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:20.180260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5563afe-ecbb-4a63-8fd0-5dab0e041858 service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:20.191182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 675/1368] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 17:51:22.730297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af93056-83f2-452b-97a9-8182d3c95612 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:22.730882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af93056-83f2-452b-97a9-8182d3c95612 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:22.730882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af93056-83f2-452b-97a9-8182d3c95612 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:22.732559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 694/1369] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:22.737752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f6134-eec7-4c2b-902d-423b224d447a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:22.738223 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f6134-eec7-4c2b-902d-423b224d447a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:22.738332 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f6134-eec7-4c2b-902d-423b224d447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:22.758682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-ea3f6134-eec7-4c2b-902d-423b224d447a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:22.760041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ea3f6134-eec7-4c2b-902d-423b224d447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:22.769767 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 676/1370] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:22.814915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af93056-83f2-452b-97a9-8182d3c95612 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:22.830887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af93056-83f2-452b-97a9-8182d3c95612 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:22.836545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3661ba1-791e-48eb-b54d-d76d391fa467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:22.837223 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3661ba1-791e-48eb-b54d-d76d391fa467 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:22.837525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3661ba1-791e-48eb-b54d-d76d391fa467 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:22.837862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3661ba1-791e-48eb-b54d-d76d391fa467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:22.847845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3661ba1-791e-48eb-b54d-d76d391fa467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:22.849706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3661ba1-791e-48eb-b54d-d76d391fa467 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:22.862044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 695/1371] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:51:22.965517 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0030a-ce1c-4251-ae61-dba6ba485644 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:22.966054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0030a-ce1c-4251-ae61-dba6ba485644 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:22.966623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0030a-ce1c-4251-ae61-dba6ba485644 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:22.968432 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 677/1372] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:22 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:22.974623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30322522-c39c-4c2b-91a9-68e552b7afb1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:22.975236 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30322522-c39c-4c2b-91a9-68e552b7afb1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:22.975520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30322522-c39c-4c2b-91a9-68e552b7afb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:22.996633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-30322522-c39c-4c2b-91a9-68e552b7afb1 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:22.997785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-30322522-c39c-4c2b-91a9-68e552b7afb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:23.007117 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 696/1373] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:23.292450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6127ac-9285-4f34-a9e6-804a70946ffa None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.293110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6127ac-9285-4f34-a9e6-804a70946ffa None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.293465 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6127ac-9285-4f34-a9e6-804a70946ffa None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.294969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 678/1374] 10.222.0.121 () {54 vars in 968 bytes} [Thu May 26 17:51:23 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:23.300718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa90dc8-bf2b-427e-876a-b4765681fee2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.301127 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa90dc8-bf2b-427e-876a-b4765681fee2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.301546 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa90dc8-bf2b-427e-876a-b4765681fee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.358914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-0fa90dc8-bf2b-427e-876a-b4765681fee2 None None] MFA Rules not processed for user `25d85d02abed40199dd2515ecf84b7fb`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:23.361972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0fa90dc8-bf2b-427e-876a-b4765681fee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:23.367248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:23.431058 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 697/1375] 10.222.0.121 () {58 vars in 1065 bytes} [Thu May 26 17:51:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:51:23.439135 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6127ac-9285-4f34-a9e6-804a70946ffa None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:23.451578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6127ac-9285-4f34-a9e6-804a70946ffa None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:23.455471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253cacf-c2c6-4551-8509-39bc70f78eb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:23.456178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253cacf-c2c6-4551-8509-39bc70f78eb6 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.456521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253cacf-c2c6-4551-8509-39bc70f78eb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.456856 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253cacf-c2c6-4551-8509-39bc70f78eb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.468241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253cacf-c2c6-4551-8509-39bc70f78eb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:23.469669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253cacf-c2c6-4551-8509-39bc70f78eb6 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:23.481900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 679/1376] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 17:51:25.378871 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b299c-96af-4e39-a392-374e9272cdf7 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:25.379514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b299c-96af-4e39-a392-374e9272cdf7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:25.379514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b299c-96af-4e39-a392-374e9272cdf7 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:25.380923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 698/1377] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:25.387962 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af6466-9c50-4219-b0e0-c2ea1e078467 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:25.388498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af6466-9c50-4219-b0e0-c2ea1e078467 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:25.388659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af6466-9c50-4219-b0e0-c2ea1e078467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:25.411209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a5af6466-9c50-4219-b0e0-c2ea1e078467 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:25.412453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a5af6466-9c50-4219-b0e0-c2ea1e078467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:25.421992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 680/1378] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:25.469481 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b299c-96af-4e39-a392-374e9272cdf7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:25.485330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b299c-96af-4e39-a392-374e9272cdf7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:25.491288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f51a05-6595-4d6d-be09-5538ab80991d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:25.491945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f51a05-6595-4d6d-be09-5538ab80991d service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:25.492224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f51a05-6595-4d6d-be09-5538ab80991d service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:25.492494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f51a05-6595-4d6d-be09-5538ab80991d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:25.502652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f51a05-6595-4d6d-be09-5538ab80991d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:25.504274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f51a05-6595-4d6d-be09-5538ab80991d service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:25.515465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 699/1379] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:51:27.996555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92e30b-12a7-4a5e-995b-2b3b01e0eb89 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.997163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92e30b-12a7-4a5e-995b-2b3b01e0eb89 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.997413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92e30b-12a7-4a5e-995b-2b3b01e0eb89 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:27.998923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 681/1380] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:27 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:28.004271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5ccd1-c8c7-451b-b50f-cd08a9d0956d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.004737 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5ccd1-c8c7-451b-b50f-cd08a9d0956d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.004847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5ccd1-c8c7-451b-b50f-cd08a9d0956d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.024507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4fe5ccd1-c8c7-451b-b50f-cd08a9d0956d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:28.026237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4fe5ccd1-c8c7-451b-b50f-cd08a9d0956d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:28.034732 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 700/1381] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:28.080333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92e30b-12a7-4a5e-995b-2b3b01e0eb89 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:28.098987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92e30b-12a7-4a5e-995b-2b3b01e0eb89 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:28.108634 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1ed85-7446-435b-b9da-be1386785bab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:28.110401 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1ed85-7446-435b-b9da-be1386785bab service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.110686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1ed85-7446-435b-b9da-be1386785bab service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.110995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1ed85-7446-435b-b9da-be1386785bab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.120519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1ed85-7446-435b-b9da-be1386785bab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:28.122096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1ed85-7446-435b-b9da-be1386785bab service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:28.132515 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 682/1382] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:51:30.570439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00568b24-cb85-4608-b857-551ac428538f None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:30.570987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00568b24-cb85-4608-b857-551ac428538f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:30.570987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00568b24-cb85-4608-b857-551ac428538f None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:30.572802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 701/1383] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:30.578611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832c365-f23c-4d60-909b-6ff2fbb90640 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:30.579127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832c365-f23c-4d60-909b-6ff2fbb90640 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:30.579416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832c365-f23c-4d60-909b-6ff2fbb90640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:30.599072 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-4832c365-f23c-4d60-909b-6ff2fbb90640 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:30.600348 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4832c365-f23c-4d60-909b-6ff2fbb90640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:30.608897 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 683/1384] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:30.653571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00568b24-cb85-4608-b857-551ac428538f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:30.675882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00568b24-cb85-4608-b857-551ac428538f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:30.682060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01797f04-561b-4d01-9df2-94ac187f882d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:30.683434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01797f04-561b-4d01-9df2-94ac187f882d service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:30.683716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01797f04-561b-4d01-9df2-94ac187f882d service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:30.684280 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01797f04-561b-4d01-9df2-94ac187f882d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:30.693605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01797f04-561b-4d01-9df2-94ac187f882d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:30.695391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01797f04-561b-4d01-9df2-94ac187f882d service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:30.706675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 702/1385] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:51:33.025059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32b274-ff7a-4ed4-bca2-fbcaa1eabe38 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:33.025756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32b274-ff7a-4ed4-bca2-fbcaa1eabe38 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:33.025989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32b274-ff7a-4ed4-bca2-fbcaa1eabe38 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:33.027275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 684/1386] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:33.033052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692018b3-71d8-42c5-a771-d9d872589ee0 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:33.033494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692018b3-71d8-42c5-a771-d9d872589ee0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:33.033587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692018b3-71d8-42c5-a771-d9d872589ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:33.052366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-692018b3-71d8-42c5-a771-d9d872589ee0 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:33.053618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-692018b3-71d8-42c5-a771-d9d872589ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:33.062440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 703/1387] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:33.109534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32b274-ff7a-4ed4-bca2-fbcaa1eabe38 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:33.125281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32b274-ff7a-4ed4-bca2-fbcaa1eabe38 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:33.130936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d0a79-7dff-4ae8-88a7-0f43f0c56a7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:33.131593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d0a79-7dff-4ae8-88a7-0f43f0c56a7b service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:33.131881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d0a79-7dff-4ae8-88a7-0f43f0c56a7b service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:33.132154 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d0a79-7dff-4ae8-88a7-0f43f0c56a7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:33.141430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d0a79-7dff-4ae8-88a7-0f43f0c56a7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:33.143035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d0a79-7dff-4ae8-88a7-0f43f0c56a7b service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:33.152883 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 685/1388] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:51:35.715739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f1888-4623-4ab1-aa54-d63f6a786235 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:35.716721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f1888-4623-4ab1-aa54-d63f6a786235 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:35.716721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f1888-4623-4ab1-aa54-d63f6a786235 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:35.717631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 704/1389] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:35.724974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755c73b-5959-4c9a-966a-30a3e8b9e6b6 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:35.725680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755c73b-5959-4c9a-966a-30a3e8b9e6b6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:35.726184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755c73b-5959-4c9a-966a-30a3e8b9e6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:35.761784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3755c73b-5959-4c9a-966a-30a3e8b9e6b6 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:35.763971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3755c73b-5959-4c9a-966a-30a3e8b9e6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:35.776403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 686/1390] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:35.824144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f1888-4623-4ab1-aa54-d63f6a786235 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:35.839368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f1888-4623-4ab1-aa54-d63f6a786235 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:35.844961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04416cb9-1f34-485e-ba25-7d6de51a158f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:35.845626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416cb9-1f34-485e-ba25-7d6de51a158f service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:35.845903 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416cb9-1f34-485e-ba25-7d6de51a158f service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:35.846173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416cb9-1f34-485e-ba25-7d6de51a158f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:35.856611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04416cb9-1f34-485e-ba25-7d6de51a158f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:35.858134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04416cb9-1f34-485e-ba25-7d6de51a158f service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:35.868556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 705/1391] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:51:38.588698 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86205031-1ed5-4cee-9032-1ad2db169e92 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:38.589327 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86205031-1ed5-4cee-9032-1ad2db169e92 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:38.589429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86205031-1ed5-4cee-9032-1ad2db169e92 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:38.590873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 687/1392] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:38.596451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ebc8b-b2f0-4635-966e-1aed6f602f97 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:38.596939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ebc8b-b2f0-4635-966e-1aed6f602f97 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:38.597060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ebc8b-b2f0-4635-966e-1aed6f602f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:38.615752 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4f9ebc8b-b2f0-4635-966e-1aed6f602f97 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:38.616952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4f9ebc8b-b2f0-4635-966e-1aed6f602f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:38.624962 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 706/1393] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:51:38.671599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86205031-1ed5-4cee-9032-1ad2db169e92 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:38.679746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=46 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:38.687281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86205031-1ed5-4cee-9032-1ad2db169e92 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:38.696595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376a469-9f21-44e0-a3bf-aaf354f94783 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:38.696595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376a469-9f21-44e0-a3bf-aaf354f94783 service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:38.696595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376a469-9f21-44e0-a3bf-aaf354f94783 service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:38.696595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376a469-9f21-44e0-a3bf-aaf354f94783 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:38.704318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376a469-9f21-44e0-a3bf-aaf354f94783 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:38.705913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376a469-9f21-44e0-a3bf-aaf354f94783 service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:38.722586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 688/1394] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:51:39.939880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebabf46-13ca-43f9-8f4e-b52655727892 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:39.941052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebabf46-13ca-43f9-8f4e-b52655727892 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:39.941052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebabf46-13ca-43f9-8f4e-b52655727892 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:39.941673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 707/1395] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:39.948247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd5b4e-9cfa-44e5-8fb0-b36fb95fd132 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:39.948784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd5b4e-9cfa-44e5-8fb0-b36fb95fd132 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:39.948891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd5b4e-9cfa-44e5-8fb0-b36fb95fd132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:39.967443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-85bd5b4e-9cfa-44e5-8fb0-b36fb95fd132 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:39.968730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-85bd5b4e-9cfa-44e5-8fb0-b36fb95fd132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:39.976888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 689/1396] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:40.040219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d609a67-0d66-401e-8da8-e76dfa18fd15 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.040661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d609a67-0d66-401e-8da8-e76dfa18fd15 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.041040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d609a67-0d66-401e-8da8-e76dfa18fd15 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.042416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 708/1397] 10.222.0.121 () {54 vars in 967 bytes} [Thu May 26 17:51:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:40.047737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b78f58-3f30-49a1-83d0-42ccf958ea2b None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.048024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b78f58-3f30-49a1-83d0-42ccf958ea2b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.048300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b78f58-3f30-49a1-83d0-42ccf958ea2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.108330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-58b78f58-3f30-49a1-83d0-42ccf958ea2b None None] MFA Rules not processed for user `683c54f489f440b9842cb94980214df1`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:40.109661 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-58b78f58-3f30-49a1-83d0-42ccf958ea2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:40.175999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 690/1398] 10.222.0.121 () {58 vars in 1064 bytes} [Thu May 26 17:51:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:40.184893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d609a67-0d66-401e-8da8-e76dfa18fd15 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:40.195825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d609a67-0d66-401e-8da8-e76dfa18fd15 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:40.200196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87dd7c6-a62f-4391-92f6-9ffa489a76e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '683c54f489f440b9842cb94980214df1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:40.200846 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dd7c6-a62f-4391-92f6-9ffa489a76e8 service glance] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.201130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dd7c6-a62f-4391-92f6-9ffa489a76e8 service glance] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.201409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dd7c6-a62f-4391-92f6-9ffa489a76e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.211404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87dd7c6-a62f-4391-92f6-9ffa489a76e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:40.213487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87dd7c6-a62f-4391-92f6-9ffa489a76e8 service glance] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:40.223607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 709/1399] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:51:40.974964 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd153a9-8640-4f84-ba9f-fd88188b732d None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.975634 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd153a9-8640-4f84-ba9f-fd88188b732d None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.975912 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd153a9-8640-4f84-ba9f-fd88188b732d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.042709 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-2bd153a9-8640-4f84-ba9f-fd88188b732d None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:41.043969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-2bd153a9-8640-4f84-ba9f-fd88188b732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:41.122207 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 691/1400] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 17:51:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:41.168307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3e793-3c21-4539-bdd8-0d026797a9f4 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.168835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3e793-3c21-4539-bdd8-0d026797a9f4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.168835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3e793-3c21-4539-bdd8-0d026797a9f4 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.170230 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 710/1401] 10.222.0.121 () {54 vars in 947 bytes} [Thu May 26 17:51:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:41.178146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2af16c-f33e-4406-922a-c1d0915d0c3b None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.178571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2af16c-f33e-4406-922a-c1d0915d0c3b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.178932 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2af16c-f33e-4406-922a-c1d0915d0c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.211851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af6fcf-077b-4773-ab3c-ae9096b6d02b None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.212109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af6fcf-077b-4773-ab3c-ae9096b6d02b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.212969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af6fcf-077b-4773-ab3c-ae9096b6d02b None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.214684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 711/1402] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:41 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:51:41.220486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f978ee-f4ad-46ed-8ef0-2ff2fd965a73 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.220772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f978ee-f4ad-46ed-8ef0-2ff2fd965a73 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.221044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f978ee-f4ad-46ed-8ef0-2ff2fd965a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.230303 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:41.233713 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5f2af16c-f33e-4406-922a-c1d0915d0c3b None None] MFA Rules not processed for user `eb9ba9c73956403ba870f6f6b0e687aa`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:41.234965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5f2af16c-f33e-4406-922a-c1d0915d0c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:41.240059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f1f978ee-f4ad-46ed-8ef0-2ff2fd965a73 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:41.243505 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f1f978ee-f4ad-46ed-8ef0-2ff2fd965a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:41.250368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 712/1403] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:51:41.296839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f978ee-f4ad-46ed-8ef0-2ff2fd965a73 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:41.309759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f978ee-f4ad-46ed-8ef0-2ff2fd965a73 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:41.315524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d87ee9-d264-4109-8554-119f79f1d169 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:41.316187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d87ee9-d264-4109-8554-119f79f1d169 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.316460 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d87ee9-d264-4109-8554-119f79f1d169 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.316724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d87ee9-d264-4109-8554-119f79f1d169 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.318450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 692/1404] 10.222.0.121 () {58 vars in 1044 bytes} [Thu May 26 17:51:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:41.326271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d87ee9-d264-4109-8554-119f79f1d169 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:41.328683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d87ee9-d264-4109-8554-119f79f1d169 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:41.329300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2af16c-f33e-4406-922a-c1d0915d0c3b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:41.341609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 713/1405] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 17:51:41.343277 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2af16c-f33e-4406-922a-c1d0915d0c3b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:41.348580 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ee7d7-2706-414e-8b04-e93fff3d0073 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:41.349266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ee7d7-2706-414e-8b04-e93fff3d0073 service swift] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.349551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ee7d7-2706-414e-8b04-e93fff3d0073 service swift] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.349828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ee7d7-2706-414e-8b04-e93fff3d0073 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.359230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ee7d7-2706-414e-8b04-e93fff3d0073 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:41.360818 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ee7d7-2706-414e-8b04-e93fff3d0073 service swift] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:41.370614 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 693/1406] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 17:51:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 17:51:43.800597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4569f28-8a97-4ec6-bca4-9fe388b5af84 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:43.801539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4569f28-8a97-4ec6-bca4-9fe388b5af84 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:43.801854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4569f28-8a97-4ec6-bca4-9fe388b5af84 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:43.803711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 714/1407] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:51:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:51:43.810722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14db16-d44c-4ad5-85dc-5afc6ef1afbc None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:43.811250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14db16-d44c-4ad5-85dc-5afc6ef1afbc None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:43.814075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14db16-d44c-4ad5-85dc-5afc6ef1afbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:43.831563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-0f14db16-d44c-4ad5-85dc-5afc6ef1afbc None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:43.832773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0f14db16-d44c-4ad5-85dc-5afc6ef1afbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:43.841389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 694/1408] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:51:43.891189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4569f28-8a97-4ec6-bca4-9fe388b5af84 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:43.911667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4569f28-8a97-4ec6-bca4-9fe388b5af84 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:43.919716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03af939-d9c4-49c6-adb2-43e290c222a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:43.920375 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03af939-d9c4-49c6-adb2-43e290c222a5 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:43.920673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03af939-d9c4-49c6-adb2-43e290c222a5 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:43.920928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03af939-d9c4-49c6-adb2-43e290c222a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:43.932465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03af939-d9c4-49c6-adb2-43e290c222a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:43.934060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03af939-d9c4-49c6-adb2-43e290c222a5 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:43.945369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 715/1409] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:51:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:51:53.825931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=39 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:54.228769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=71 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:07.185132 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc884b34-5602-426f-98e3-b74c2fe5983d None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:07.185769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc884b34-5602-426f-98e3-b74c2fe5983d None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:07.185769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc884b34-5602-426f-98e3-b74c2fe5983d None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:07.187092 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 695/1410] 10.222.0.121 () {54 vars in 923 bytes} [Thu May 26 17:52:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:07.192604 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574382bc-de6a-44f2-bf65-0e025929edb4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:07.193109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574382bc-de6a-44f2-bf65-0e025929edb4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:07.193218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574382bc-de6a-44f2-bf65-0e025929edb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:07.212980 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-574382bc-de6a-44f2-bf65-0e025929edb4 None None] MFA Rules not processed for user `7847d2c119474d49bef6029d51dbefdb`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:07.214188 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-574382bc-de6a-44f2-bf65-0e025929edb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:07.222905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 716/1411] 10.222.0.121 () {58 vars in 1020 bytes} [Thu May 26 17:52:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:07.238879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3467b-449c-493d-9b2b-1bfe6a498345 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:07.239084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3467b-449c-493d-9b2b-1bfe6a498345 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:07.239343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3467b-449c-493d-9b2b-1bfe6a498345 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:07.240767 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 696/1412] 10.222.0.121 () {54 vars in 923 bytes} [Thu May 26 17:52:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:07.245913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3b313-c886-4eab-9908-30b3aa4fead7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:07.246200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3b313-c886-4eab-9908-30b3aa4fead7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:07.246482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3b313-c886-4eab-9908-30b3aa4fead7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:07.266294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-0be3b313-c886-4eab-9908-30b3aa4fead7 None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:07.267551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0be3b313-c886-4eab-9908-30b3aa4fead7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:07.278292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 717/1413] 10.222.0.121 () {58 vars in 1020 bytes} [Thu May 26 17:52:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:10.097598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de19f6-1558-49de-8a78-9bb3ecdc02be None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:10.098131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de19f6-1558-49de-8a78-9bb3ecdc02be None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:10.098131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de19f6-1558-49de-8a78-9bb3ecdc02be None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:10.099529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:52:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:10.106052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aaa607-338c-4659-a115-daceb16de273 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:10.106932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aaa607-338c-4659-a115-daceb16de273 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:10.107106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aaa607-338c-4659-a115-daceb16de273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:10.128114 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d0aaa607-338c-4659-a115-daceb16de273 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:10.129376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d0aaa607-338c-4659-a115-daceb16de273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:10.138906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:10.185702 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de19f6-1558-49de-8a78-9bb3ecdc02be None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:10.203492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de19f6-1558-49de-8a78-9bb3ecdc02be None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:10.209512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e18c1-e854-4cf8-b3f1-1a40c23606e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:10.210179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e18c1-e854-4cf8-b3f1-1a40c23606e5 service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:10.213139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e18c1-e854-4cf8-b3f1-1a40c23606e5 service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:10.213139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e18c1-e854-4cf8-b3f1-1a40c23606e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:10.224009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e18c1-e854-4cf8-b3f1-1a40c23606e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:10.225689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e18c1-e854-4cf8-b3f1-1a40c23606e5 service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:10.236645 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 698/1416] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:11.266607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76017d0-6c3c-4fda-90bd-b9f5ed5ff749 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:11.267233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76017d0-6c3c-4fda-90bd-b9f5ed5ff749 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:11.267357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76017d0-6c3c-4fda-90bd-b9f5ed5ff749 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:11.268643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:52:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:11.275005 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d3aed-e5c5-43a4-b162-cc1eff922ee6 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:11.275514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d3aed-e5c5-43a4-b162-cc1eff922ee6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:11.275606 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d3aed-e5c5-43a4-b162-cc1eff922ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:11.295055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5a2d3aed-e5c5-43a4-b162-cc1eff922ee6 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:11.296314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5a2d3aed-e5c5-43a4-b162-cc1eff922ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:11.304165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:52:11.481899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76017d0-6c3c-4fda-90bd-b9f5ed5ff749 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:11.498924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76017d0-6c3c-4fda-90bd-b9f5ed5ff749 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:11.504325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02a228-1584-4247-bf9a-79a8e4d325a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:11.504988 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02a228-1584-4247-bf9a-79a8e4d325a4 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:11.505266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02a228-1584-4247-bf9a-79a8e4d325a4 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:11.505539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02a228-1584-4247-bf9a-79a8e4d325a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:11.514946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02a228-1584-4247-bf9a-79a8e4d325a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:11.516755 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02a228-1584-4247-bf9a-79a8e4d325a4 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:11.527567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 720/1419] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:12.493184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dced44-8b17-4371-bbf9-ae80cc36e5c9 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:12.493830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dced44-8b17-4371-bbf9-ae80cc36e5c9 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:12.493830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dced44-8b17-4371-bbf9-ae80cc36e5c9 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:12.495290 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:52:12 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:12.500465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090aaa9-1868-4b73-9da9-800c6126d372 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:12.500895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090aaa9-1868-4b73-9da9-800c6126d372 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:12.501024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090aaa9-1868-4b73-9da9-800c6126d372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:12.519575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c090aaa9-1868-4b73-9da9-800c6126d372 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:12.520804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c090aaa9-1868-4b73-9da9-800c6126d372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:12.529412 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:12.683728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dced44-8b17-4371-bbf9-ae80cc36e5c9 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:12.698545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dced44-8b17-4371-bbf9-ae80cc36e5c9 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:12.704155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756ee5d-d1bb-44fd-86bc-5ff57f66ba8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:12.704807 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756ee5d-d1bb-44fd-86bc-5ff57f66ba8b service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:12.705089 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756ee5d-d1bb-44fd-86bc-5ff57f66ba8b service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:12.705364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756ee5d-d1bb-44fd-86bc-5ff57f66ba8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:12.714910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756ee5d-d1bb-44fd-86bc-5ff57f66ba8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:12.716496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756ee5d-d1bb-44fd-86bc-5ff57f66ba8b service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:12.727773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 701/1422] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:13.672382 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2afa7a-e2a0-430e-9501-cdaccdb20a2a None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:13.672892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2afa7a-e2a0-430e-9501-cdaccdb20a2a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:13.672892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2afa7a-e2a0-430e-9501-cdaccdb20a2a None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:13.674215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:52:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:13.679722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286db9fe-020c-482c-9aed-f840038880ca None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:13.680215 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286db9fe-020c-482c-9aed-f840038880ca None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:13.680309 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286db9fe-020c-482c-9aed-f840038880ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:13.698888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-286db9fe-020c-482c-9aed-f840038880ca None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:13.700165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-286db9fe-020c-482c-9aed-f840038880ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:13.707998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:13.859850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2afa7a-e2a0-430e-9501-cdaccdb20a2a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:13.881621 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2afa7a-e2a0-430e-9501-cdaccdb20a2a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:13.888208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccbd6a-260b-40ce-ac67-0fb918ef9e45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:13.888855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccbd6a-260b-40ce-ac67-0fb918ef9e45 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:13.889165 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccbd6a-260b-40ce-ac67-0fb918ef9e45 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:13.889431 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccbd6a-260b-40ce-ac67-0fb918ef9e45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:13.898615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccbd6a-260b-40ce-ac67-0fb918ef9e45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:13.900318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccbd6a-260b-40ce-ac67-0fb918ef9e45 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:13.910195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 723/1425] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:14.849218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3032f-b721-44e2-9ffe-4a94849c969c None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:14.849830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3032f-b721-44e2-9ffe-4a94849c969c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:14.849830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3032f-b721-44e2-9ffe-4a94849c969c None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:14.851263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:52:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:14.856532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37e25c-a9ac-4374-9adc-4c847b21d5bd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:14.857001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37e25c-a9ac-4374-9adc-4c847b21d5bd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:14.857119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37e25c-a9ac-4374-9adc-4c847b21d5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:14.880934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8c37e25c-a9ac-4374-9adc-4c847b21d5bd None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:14.882917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8c37e25c-a9ac-4374-9adc-4c847b21d5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:14.891271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:15.048061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3032f-b721-44e2-9ffe-4a94849c969c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:15.063315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3032f-b721-44e2-9ffe-4a94849c969c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:15.070710 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127436f-c099-4175-8b61-fe4b6a1b96d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:15.071393 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127436f-c099-4175-8b61-fe4b6a1b96d3 service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:15.071673 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127436f-c099-4175-8b61-fe4b6a1b96d3 service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:15.071938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127436f-c099-4175-8b61-fe4b6a1b96d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:15.080579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127436f-c099-4175-8b61-fe4b6a1b96d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:15.082130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127436f-c099-4175-8b61-fe4b6a1b96d3 service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:15.091617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 704/1428] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:16.022366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b9a69-a7ec-4820-83b7-bcf1a646a9d9 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:16.022965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b9a69-a7ec-4820-83b7-bcf1a646a9d9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:16.022965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b9a69-a7ec-4820-83b7-bcf1a646a9d9 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:16.024352 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:52:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:16.029862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39629a6-4137-4ef1-a611-03efc67788f6 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:16.030314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39629a6-4137-4ef1-a611-03efc67788f6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:16.030432 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39629a6-4137-4ef1-a611-03efc67788f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:16.050915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c39629a6-4137-4ef1-a611-03efc67788f6 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:16.052098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c39629a6-4137-4ef1-a611-03efc67788f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:16.061118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:16.213609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b9a69-a7ec-4820-83b7-bcf1a646a9d9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:16.228776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b9a69-a7ec-4820-83b7-bcf1a646a9d9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:16.234059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875c7fd-5a79-4c4c-a2e3-d6275d1c4124 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:16.234704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875c7fd-5a79-4c4c-a2e3-d6275d1c4124 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:16.235156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875c7fd-5a79-4c4c-a2e3-d6275d1c4124 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:16.236010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875c7fd-5a79-4c4c-a2e3-d6275d1c4124 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:16.244699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875c7fd-5a79-4c4c-a2e3-d6275d1c4124 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:16.250360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875c7fd-5a79-4c4c-a2e3-d6275d1c4124 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:16.268799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 726/1431] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:17.210607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ba871-9cac-4dfd-a334-904ed62cc438 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:17.211306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ba871-9cac-4dfd-a334-904ed62cc438 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:17.211306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ba871-9cac-4dfd-a334-904ed62cc438 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:17.212740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:52:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:17.220324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87c759-1cb4-44dc-b107-18222b5f8e78 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:17.220868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87c759-1cb4-44dc-b107-18222b5f8e78 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:17.220868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87c759-1cb4-44dc-b107-18222b5f8e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:17.240235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ad87c759-1cb4-44dc-b107-18222b5f8e78 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:17.242337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ad87c759-1cb4-44dc-b107-18222b5f8e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:17.251568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:17.409560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ba871-9cac-4dfd-a334-904ed62cc438 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:17.425342 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ba871-9cac-4dfd-a334-904ed62cc438 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:17.432785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8debd-08d3-419d-a452-01a2c51dabf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:17.437549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8debd-08d3-419d-a452-01a2c51dabf9 service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:17.444382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8debd-08d3-419d-a452-01a2c51dabf9 service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:17.444382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8debd-08d3-419d-a452-01a2c51dabf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:17.447625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8debd-08d3-419d-a452-01a2c51dabf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:17.449233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8debd-08d3-419d-a452-01a2c51dabf9 service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:17.461105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 707/1434] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:18.423510 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7226f-7055-4d54-accc-8fa9347ad1b4 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:18.424296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7226f-7055-4d54-accc-8fa9347ad1b4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:18.424296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7226f-7055-4d54-accc-8fa9347ad1b4 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:18.425472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:52:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:18.432976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670db570-d573-4ede-9183-3bce9250e1de None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:18.433788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670db570-d573-4ede-9183-3bce9250e1de None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:18.433788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670db570-d573-4ede-9183-3bce9250e1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:18.463156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-670db570-d573-4ede-9183-3bce9250e1de None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:18.464382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-670db570-d573-4ede-9183-3bce9250e1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:18.479214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:18.640992 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7226f-7055-4d54-accc-8fa9347ad1b4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:18.656133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7226f-7055-4d54-accc-8fa9347ad1b4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:18.661733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cca741-69f8-4254-a879-df7b417e516b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:18.662426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca741-69f8-4254-a879-df7b417e516b service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:18.662707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca741-69f8-4254-a879-df7b417e516b service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:18.664310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca741-69f8-4254-a879-df7b417e516b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:18.673346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cca741-69f8-4254-a879-df7b417e516b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:18.675119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cca741-69f8-4254-a879-df7b417e516b service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:18.685231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 729/1437] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:19.596003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f29cf3-e3fe-472e-9d54-627bfbae4cda None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:19.596610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f29cf3-e3fe-472e-9d54-627bfbae4cda None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:19.596610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f29cf3-e3fe-472e-9d54-627bfbae4cda None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:19.598385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:52:19 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:19.603827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d2e1c-f2d0-4704-90b6-84f986e3b13c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:19.604247 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d2e1c-f2d0-4704-90b6-84f986e3b13c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:19.604400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d2e1c-f2d0-4704-90b6-84f986e3b13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:19.624006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ec6d2e1c-f2d0-4704-90b6-84f986e3b13c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:19.625209 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ec6d2e1c-f2d0-4704-90b6-84f986e3b13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:19.633774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:19.783855 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f29cf3-e3fe-472e-9d54-627bfbae4cda None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:19.800655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f29cf3-e3fe-472e-9d54-627bfbae4cda None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:19.807159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23461a1f-eda6-43f8-bc8c-b274d6cd66af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:19.807883 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23461a1f-eda6-43f8-bc8c-b274d6cd66af service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:19.808123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23461a1f-eda6-43f8-bc8c-b274d6cd66af service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:19.808380 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23461a1f-eda6-43f8-bc8c-b274d6cd66af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:19.817713 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23461a1f-eda6-43f8-bc8c-b274d6cd66af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:19.825244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23461a1f-eda6-43f8-bc8c-b274d6cd66af service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:19.830159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 710/1440] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:21.970787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f24fb-3895-415c-bb09-7bb0f7dd2b63 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:21.971393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f24fb-3895-415c-bb09-7bb0f7dd2b63 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:21.971393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f24fb-3895-415c-bb09-7bb0f7dd2b63 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:21.972718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:52:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:21.978730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58540fef-e267-4017-8358-021e02834d5b None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:21.979187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58540fef-e267-4017-8358-021e02834d5b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:21.979355 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58540fef-e267-4017-8358-021e02834d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:21.998103 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-58540fef-e267-4017-8358-021e02834d5b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:21.999337 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-58540fef-e267-4017-8358-021e02834d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:22.007225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:22.046273 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f24fb-3895-415c-bb09-7bb0f7dd2b63 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:22.057803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f24fb-3895-415c-bb09-7bb0f7dd2b63 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:22.061353 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7744cd-8eec-41e1-91a1-8cdbb59bc619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:22.062001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7744cd-8eec-41e1-91a1-8cdbb59bc619 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:22.062283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7744cd-8eec-41e1-91a1-8cdbb59bc619 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:22.062558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7744cd-8eec-41e1-91a1-8cdbb59bc619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:22.072083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7744cd-8eec-41e1-91a1-8cdbb59bc619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:22.073537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7744cd-8eec-41e1-91a1-8cdbb59bc619 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:22.082931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 732/1443] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:24.462147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c14532-20cf-49e4-bda6-a8cba5375d3b None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:24.462720 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c14532-20cf-49e4-bda6-a8cba5375d3b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:24.462960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c14532-20cf-49e4-bda6-a8cba5375d3b None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:24.464623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:52:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:24.471561 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cb4aa-6258-40ff-90f3-48ece407e44f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:24.472113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cb4aa-6258-40ff-90f3-48ece407e44f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:24.472113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cb4aa-6258-40ff-90f3-48ece407e44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:24.490204 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-098cb4aa-6258-40ff-90f3-48ece407e44f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:24.491444 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-098cb4aa-6258-40ff-90f3-48ece407e44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:24.499886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:24.537814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c14532-20cf-49e4-bda6-a8cba5375d3b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:24.549539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c14532-20cf-49e4-bda6-a8cba5375d3b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:24.553147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646fdb3-261b-49de-9d42-40b59462af65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:24.553803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646fdb3-261b-49de-9d42-40b59462af65 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:24.554174 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646fdb3-261b-49de-9d42-40b59462af65 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:24.554601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646fdb3-261b-49de-9d42-40b59462af65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:24.564790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a646fdb3-261b-49de-9d42-40b59462af65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:24.566199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a646fdb3-261b-49de-9d42-40b59462af65 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:24.575851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 713/1446] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:27.219071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cad0de-70c2-4236-b354-26c2399c02e5 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:27.219705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cad0de-70c2-4236-b354-26c2399c02e5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:27.219705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cad0de-70c2-4236-b354-26c2399c02e5 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:27.221146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:52:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:27.226615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c318819-f9d3-4c45-b62d-1294ffd2e833 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:27.227239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c318819-f9d3-4c45-b62d-1294ffd2e833 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:27.227239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c318819-f9d3-4c45-b62d-1294ffd2e833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:27.244984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-0c318819-f9d3-4c45-b62d-1294ffd2e833 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:27.246212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0c318819-f9d3-4c45-b62d-1294ffd2e833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:27.254228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:52:27.294051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cad0de-70c2-4236-b354-26c2399c02e5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:27.305305 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cad0de-70c2-4236-b354-26c2399c02e5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:27.308868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c65b5-67f6-4edf-ab31-5f738cd1c222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:27.309506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c65b5-67f6-4edf-ab31-5f738cd1c222 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:27.309798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c65b5-67f6-4edf-ab31-5f738cd1c222 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:27.310054 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c65b5-67f6-4edf-ab31-5f738cd1c222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:27.319563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c65b5-67f6-4edf-ab31-5f738cd1c222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:27.321882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c65b5-67f6-4edf-ab31-5f738cd1c222 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:27.332575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 735/1449] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:29.689540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c9c43-8f59-4ef5-a2ae-af82c777d16f None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:29.690136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c9c43-8f59-4ef5-a2ae-af82c777d16f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:29.690136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c9c43-8f59-4ef5-a2ae-af82c777d16f None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:29.691488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:52:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:29.696828 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e72303-f480-4e02-8102-bdbce806a093 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:29.697314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e72303-f480-4e02-8102-bdbce806a093 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:29.697424 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e72303-f480-4e02-8102-bdbce806a093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:29.715773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-73e72303-f480-4e02-8102-bdbce806a093 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:29.716967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-73e72303-f480-4e02-8102-bdbce806a093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:29.724652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:29.771610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c9c43-8f59-4ef5-a2ae-af82c777d16f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:29.775890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c9c43-8f59-4ef5-a2ae-af82c777d16f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:29.780075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4057210-e5e4-4619-8c94-d7df3153c0de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:29.780769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4057210-e5e4-4619-8c94-d7df3153c0de service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:29.781101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4057210-e5e4-4619-8c94-d7df3153c0de service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:29.781414 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4057210-e5e4-4619-8c94-d7df3153c0de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:29.791828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4057210-e5e4-4619-8c94-d7df3153c0de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:29.793529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4057210-e5e4-4619-8c94-d7df3153c0de service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:29.803350 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 716/1452] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 17:52:32.214902 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e07a1-c398-427e-99bf-7e4a67800b0d None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:32.215500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e07a1-c398-427e-99bf-7e4a67800b0d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:32.215500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e07a1-c398-427e-99bf-7e4a67800b0d None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:32.216807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:52:32 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:32.223065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b2655-b3af-46e6-915b-0a8c38f22a62 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:32.223878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b2655-b3af-46e6-915b-0a8c38f22a62 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:32.224041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b2655-b3af-46e6-915b-0a8c38f22a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:32.243423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-8f1b2655-b3af-46e6-915b-0a8c38f22a62 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:32.244620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-8f1b2655-b3af-46e6-915b-0a8c38f22a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:32.252272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:32.292644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e07a1-c398-427e-99bf-7e4a67800b0d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:32.303096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e07a1-c398-427e-99bf-7e4a67800b0d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:32.306547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2052c5-1982-4805-b0f3-bc46658539e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:32.307214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2052c5-1982-4805-b0f3-bc46658539e5 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:32.307491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2052c5-1982-4805-b0f3-bc46658539e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:32.307761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2052c5-1982-4805-b0f3-bc46658539e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:32.316882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2052c5-1982-4805-b0f3-bc46658539e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:32.318301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2052c5-1982-4805-b0f3-bc46658539e5 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:32.327948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 738/1455] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:34.633189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea5ac3-8d2c-4c0a-b5dd-3bec209b91d3 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:34.633799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea5ac3-8d2c-4c0a-b5dd-3bec209b91d3 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:34.633799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea5ac3-8d2c-4c0a-b5dd-3bec209b91d3 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:34.635217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:52:34 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:34.640706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e89c4-95d0-45d3-885b-a4263c80717d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:34.641231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e89c4-95d0-45d3-885b-a4263c80717d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:34.641336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e89c4-95d0-45d3-885b-a4263c80717d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:34.660364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d01e89c4-95d0-45d3-885b-a4263c80717d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:34.661606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d01e89c4-95d0-45d3-885b-a4263c80717d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:34.670054 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:34.709352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea5ac3-8d2c-4c0a-b5dd-3bec209b91d3 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:34.715639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=64 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:34.721181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea5ac3-8d2c-4c0a-b5dd-3bec209b91d3 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:34.724629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cf584-ea48-42eb-9d40-fe0337018a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:34.725292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cf584-ea48-42eb-9d40-fe0337018a49 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:34.727217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cf584-ea48-42eb-9d40-fe0337018a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:34.727217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cf584-ea48-42eb-9d40-fe0337018a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:34.735197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cf584-ea48-42eb-9d40-fe0337018a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:34.736733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cf584-ea48-42eb-9d40-fe0337018a49 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:34.746708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 719/1458] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:37.055341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3ac42-7edc-4cbe-92bf-22e88a779c23 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:37.055912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3ac42-7edc-4cbe-92bf-22e88a779c23 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:37.056018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3ac42-7edc-4cbe-92bf-22e88a779c23 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:37.057386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:52:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:37.062996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d4fb6-72d9-40b9-ad42-5b8c95015f8c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:37.063479 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d4fb6-72d9-40b9-ad42-5b8c95015f8c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:37.063588 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d4fb6-72d9-40b9-ad42-5b8c95015f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:37.083246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f93d4fb6-72d9-40b9-ad42-5b8c95015f8c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:37.084730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f93d4fb6-72d9-40b9-ad42-5b8c95015f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:37.093459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:37.131508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3ac42-7edc-4cbe-92bf-22e88a779c23 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:37.137660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=73 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:37.143472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3ac42-7edc-4cbe-92bf-22e88a779c23 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:37.147093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3314f-95f9-41f6-b05d-e743ca1dbaea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:37.148359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3314f-95f9-41f6-b05d-e743ca1dbaea service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:37.148628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3314f-95f9-41f6-b05d-e743ca1dbaea service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:37.148897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3314f-95f9-41f6-b05d-e743ca1dbaea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:37.160186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3314f-95f9-41f6-b05d-e743ca1dbaea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:37.163627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3314f-95f9-41f6-b05d-e743ca1dbaea service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:37.172283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 741/1461] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:39.438140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8af87-c39b-45fd-bdc7-7dc4dca6c598 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:39.438710 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8af87-c39b-45fd-bdc7-7dc4dca6c598 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:39.438710 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8af87-c39b-45fd-bdc7-7dc4dca6c598 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:39.440084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:52:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:39.445505 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f01a53-6875-45fd-ad01-506d12040a3d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:39.451908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f01a53-6875-45fd-ad01-506d12040a3d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:39.451908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f01a53-6875-45fd-ad01-506d12040a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:39.464394 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-83f01a53-6875-45fd-ad01-506d12040a3d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:39.465597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-83f01a53-6875-45fd-ad01-506d12040a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:39.473495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:39.510943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8af87-c39b-45fd-bdc7-7dc4dca6c598 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:39.523496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8af87-c39b-45fd-bdc7-7dc4dca6c598 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:39.526960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128314f-ce5a-487d-8834-7549a4e000e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:39.527610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128314f-ce5a-487d-8834-7549a4e000e6 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:39.527912 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128314f-ce5a-487d-8834-7549a4e000e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:39.528179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128314f-ce5a-487d-8834-7549a4e000e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:39.538381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128314f-ce5a-487d-8834-7549a4e000e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:39.539907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128314f-ce5a-487d-8834-7549a4e000e6 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:39.549309 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 722/1464] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:41.791939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14035b07-91c3-44d1-83e4-e0cdca2c7185 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:41.792515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14035b07-91c3-44d1-83e4-e0cdca2c7185 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:41.792515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14035b07-91c3-44d1-83e4-e0cdca2c7185 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:41.793957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:52:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:41.799561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b7e57-cb2e-4c3c-bf92-0d84786479ba None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:41.800018 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b7e57-cb2e-4c3c-bf92-0d84786479ba None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:41.800116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b7e57-cb2e-4c3c-bf92-0d84786479ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:41.817977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-000b7e57-cb2e-4c3c-bf92-0d84786479ba None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:41.819434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-000b7e57-cb2e-4c3c-bf92-0d84786479ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:41.827210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:41.863887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14035b07-91c3-44d1-83e4-e0cdca2c7185 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:41.874707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14035b07-91c3-44d1-83e4-e0cdca2c7185 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:41.878320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96520a5-6f5e-4bab-829c-bd9761b89d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:41.879003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96520a5-6f5e-4bab-829c-bd9761b89d54 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:41.879285 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96520a5-6f5e-4bab-829c-bd9761b89d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:41.879562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96520a5-6f5e-4bab-829c-bd9761b89d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:41.888530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96520a5-6f5e-4bab-829c-bd9761b89d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:41.889959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96520a5-6f5e-4bab-829c-bd9761b89d54 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:41.899826 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 744/1467] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:44.155466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c84b6-b02d-497c-9b39-130022a39a6f None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:44.156022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c84b6-b02d-497c-9b39-130022a39a6f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:44.156022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c84b6-b02d-497c-9b39-130022a39a6f None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:44.157432 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:52:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:52:44.162864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95439041-4f8d-4438-8fae-de8305ec0db9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:44.163565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95439041-4f8d-4438-8fae-de8305ec0db9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:44.163565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95439041-4f8d-4438-8fae-de8305ec0db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:44.182575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-95439041-4f8d-4438-8fae-de8305ec0db9 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:44.183802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-95439041-4f8d-4438-8fae-de8305ec0db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:44.191956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:52:44.228328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c84b6-b02d-497c-9b39-130022a39a6f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:44.239293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c84b6-b02d-497c-9b39-130022a39a6f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:44.243279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a0328-4b2e-443a-a720-35e69ae7edc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:44.243918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a0328-4b2e-443a-a720-35e69ae7edc5 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:44.244200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a0328-4b2e-443a-a720-35e69ae7edc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:44.244472 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a0328-4b2e-443a-a720-35e69ae7edc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:44.253370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a0328-4b2e-443a-a720-35e69ae7edc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:44.254780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a0328-4b2e-443a-a720-35e69ae7edc5 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:44.264987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 725/1470] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:52:54.176845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=17 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:54.260721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=23 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:53:18.562153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f091c-ac1c-48b6-a43d-bece7602bf3e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:18.562776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f091c-ac1c-48b6-a43d-bece7602bf3e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:18.562776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f091c-ac1c-48b6-a43d-bece7602bf3e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:18.564158 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 746/1471] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:53:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:53:18.570786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b57520-acfd-4873-acfe-24280d86e726 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:18.571330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b57520-acfd-4873-acfe-24280d86e726 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:18.571430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b57520-acfd-4873-acfe-24280d86e726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:18.589623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-03b57520-acfd-4873-acfe-24280d86e726 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:53:18.590809 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-03b57520-acfd-4873-acfe-24280d86e726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:53:18.599362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 726/1472] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:53:18.647887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f33e60-9245-423e-ad4b-bd80ea9645a4 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:18.648172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f33e60-9245-423e-ad4b-bd80ea9645a4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:18.648445 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f33e60-9245-423e-ad4b-bd80ea9645a4 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:18.649810 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 747/1473] 10.222.0.121 () {54 vars in 967 bytes} [Thu May 26 17:53:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:53:18.655396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6212a-674b-4303-bed0-1f9356065b45 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:18.655682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6212a-674b-4303-bed0-1f9356065b45 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:18.655954 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6212a-674b-4303-bed0-1f9356065b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:18.680200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-6ec6212a-674b-4303-bed0-1f9356065b45 None None] MFA Rules not processed for user `683c54f489f440b9842cb94980214df1`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:53:18.681406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6ec6212a-674b-4303-bed0-1f9356065b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:53:18.687839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 727/1474] 10.222.0.121 () {58 vars in 1064 bytes} [Thu May 26 17:53:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:53:18.695229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f33e60-9245-423e-ad4b-bd80ea9645a4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:18.706858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f33e60-9245-423e-ad4b-bd80ea9645a4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:18.711738 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c9583-83b9-4008-bf21-544f5f987100 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '683c54f489f440b9842cb94980214df1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:18.712387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c9583-83b9-4008-bf21-544f5f987100 service glance] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:18.712670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c9583-83b9-4008-bf21-544f5f987100 service glance] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:18.712951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c9583-83b9-4008-bf21-544f5f987100 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:18.722350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c9583-83b9-4008-bf21-544f5f987100 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:53:18.725208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c9583-83b9-4008-bf21-544f5f987100 service glance] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:53:18.734937 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 748/1475] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:53:21.115354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5d4a8-07e4-4156-8aad-1b11e330f521 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:21.115919 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5d4a8-07e4-4156-8aad-1b11e330f521 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:21.115919 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5d4a8-07e4-4156-8aad-1b11e330f521 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:21.117360 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 728/1476] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:53:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:53:21.124429 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da390b-97ef-4a79-85bc-178108fb63af None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:21.124979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da390b-97ef-4a79-85bc-178108fb63af None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:21.125093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da390b-97ef-4a79-85bc-178108fb63af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:21.150588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-80da390b-97ef-4a79-85bc-178108fb63af None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:53:21.153135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-80da390b-97ef-4a79-85bc-178108fb63af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:53:21.164882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 749/1477] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:53:21.190385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5d4a8-07e4-4156-8aad-1b11e330f521 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:21.205339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5d4a8-07e4-4156-8aad-1b11e330f521 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:21.210683 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497546a9-999f-45a7-9fed-e1b8eadf83c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:21.211372 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497546a9-999f-45a7-9fed-e1b8eadf83c0 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:21.211647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497546a9-999f-45a7-9fed-e1b8eadf83c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:21.211921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497546a9-999f-45a7-9fed-e1b8eadf83c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:21.212648 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497546a9-999f-45a7-9fed-e1b8eadf83c0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:53:21.214409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: WARNING py.warnings [None req-497546a9-999f-45a7-9fed-e1b8eadf83c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 17:53:21.214409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: warnings.warn(msg) May 26 17:53:21.214409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: May 26 17:53:21.215564 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497546a9-999f-45a7-9fed-e1b8eadf83c0 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:53:21.236299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 729/1478] 10.222.0.121 () {56 vars in 1113 bytes} [Thu May 26 17:53:21 2022] GET /identity/v3/projects => generated 2562 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:53:23.547660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86c458-33c0-4194-9685-94a4e3bc151a None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:23.548212 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86c458-33c0-4194-9685-94a4e3bc151a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:23.548212 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86c458-33c0-4194-9685-94a4e3bc151a None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:23.549618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 750/1479] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:53:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:53:23.555228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df02659-d214-4495-8b8e-ede81c33dfc6 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:23.555690 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df02659-d214-4495-8b8e-ede81c33dfc6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:23.555890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df02659-d214-4495-8b8e-ede81c33dfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:23.574939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-2df02659-d214-4495-8b8e-ede81c33dfc6 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:53:23.576158 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-2df02659-d214-4495-8b8e-ede81c33dfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:53:23.585317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 730/1480] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:53:23.631134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86c458-33c0-4194-9685-94a4e3bc151a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:23.646536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86c458-33c0-4194-9685-94a4e3bc151a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:23.652024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db41ded-ea1a-4953-974e-f1ca83a8d372 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:23.652925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db41ded-ea1a-4953-974e-f1ca83a8d372 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:23.653201 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db41ded-ea1a-4953-974e-f1ca83a8d372 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:23.653472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db41ded-ea1a-4953-974e-f1ca83a8d372 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:23.662928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db41ded-ea1a-4953-974e-f1ca83a8d372 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:53:23.664553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db41ded-ea1a-4953-974e-f1ca83a8d372 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:53:23.674704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 751/1481] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:53:26.143517 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfe801-754e-476d-ba24-8b5cd32b93c1 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:26.144103 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfe801-754e-476d-ba24-8b5cd32b93c1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:26.144103 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfe801-754e-476d-ba24-8b5cd32b93c1 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:26.146377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 731/1482] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:53:26 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:53:26.151814 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c136d9-b492-4179-af2b-e4a242ef5bea None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:26.152253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c136d9-b492-4179-af2b-e4a242ef5bea None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:26.152362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c136d9-b492-4179-af2b-e4a242ef5bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:26.171351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e7c136d9-b492-4179-af2b-e4a242ef5bea None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:53:26.172550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e7c136d9-b492-4179-af2b-e4a242ef5bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:53:26.180549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 752/1483] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:53:26.218166 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfe801-754e-476d-ba24-8b5cd32b93c1 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:26.231564 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfe801-754e-476d-ba24-8b5cd32b93c1 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:26.235331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2bbf8-6b91-4a6e-86a7-8d5270cbc3e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:26.235982 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2bbf8-6b91-4a6e-86a7-8d5270cbc3e8 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:26.236262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2bbf8-6b91-4a6e-86a7-8d5270cbc3e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:26.236530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2bbf8-6b91-4a6e-86a7-8d5270cbc3e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:26.246718 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2bbf8-6b91-4a6e-86a7-8d5270cbc3e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:53:26.248170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2bbf8-6b91-4a6e-86a7-8d5270cbc3e8 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:53:26.257916 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 732/1484] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:53:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:53:28.628032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dd1b7-296e-4188-8aa5-4c9a50e3e5bf None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:28.628660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dd1b7-296e-4188-8aa5-4c9a50e3e5bf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:28.628660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dd1b7-296e-4188-8aa5-4c9a50e3e5bf None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:28.630089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 753/1485] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:53:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:53:28.635915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d143f-ecde-43b4-8d93-0a58273f8ea2 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:28.636452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d143f-ecde-43b4-8d93-0a58273f8ea2 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:28.636452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d143f-ecde-43b4-8d93-0a58273f8ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:28.654854 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-425d143f-ecde-43b4-8d93-0a58273f8ea2 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:53:28.656053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-425d143f-ecde-43b4-8d93-0a58273f8ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:53:28.664482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 733/1486] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:53:28.702467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dd1b7-296e-4188-8aa5-4c9a50e3e5bf None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:28.713560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dd1b7-296e-4188-8aa5-4c9a50e3e5bf None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:28.717142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07055356-d0e7-4774-b0fd-ed50d2b8093a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:28.717829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07055356-d0e7-4774-b0fd-ed50d2b8093a service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:28.718109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07055356-d0e7-4774-b0fd-ed50d2b8093a service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:28.718501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07055356-d0e7-4774-b0fd-ed50d2b8093a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:28.728684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07055356-d0e7-4774-b0fd-ed50d2b8093a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:53:28.730251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07055356-d0e7-4774-b0fd-ed50d2b8093a service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:53:28.740867 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 754/1487] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:53:31.188370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea38f61-a48d-4a02-a988-11dad896769d None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:31.188945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea38f61-a48d-4a02-a988-11dad896769d None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:31.188945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea38f61-a48d-4a02-a988-11dad896769d None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:31.190358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 734/1488] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:53:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:53:31.196332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27c5f0-6a18-4cad-b833-1bb4d7c9975e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:31.196888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27c5f0-6a18-4cad-b833-1bb4d7c9975e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:31.196888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27c5f0-6a18-4cad-b833-1bb4d7c9975e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:31.215205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2b27c5f0-6a18-4cad-b833-1bb4d7c9975e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:53:31.216414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2b27c5f0-6a18-4cad-b833-1bb4d7c9975e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:53:31.225132 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 755/1489] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:53:31.265144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea38f61-a48d-4a02-a988-11dad896769d None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:31.276614 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea38f61-a48d-4a02-a988-11dad896769d None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:31.280557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a8f17-760d-41e8-b800-7ee5571736bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:31.281191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a8f17-760d-41e8-b800-7ee5571736bb service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:31.281467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a8f17-760d-41e8-b800-7ee5571736bb service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:31.281737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a8f17-760d-41e8-b800-7ee5571736bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:31.291589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a8f17-760d-41e8-b800-7ee5571736bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:53:31.292941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a8f17-760d-41e8-b800-7ee5571736bb service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:53:31.308241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 735/1490] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:53:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:53:33.991931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7660c-4e87-4dd2-93ce-6a78f38aabbc None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:33.992513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7660c-4e87-4dd2-93ce-6a78f38aabbc None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:33.992513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7660c-4e87-4dd2-93ce-6a78f38aabbc None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:33.993934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 756/1491] 10.222.0.121 () {54 vars in 931 bytes} [Thu May 26 17:53:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:53:33.999218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c096ed-1fcc-4b62-872e-8e2c2e124d02 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:33.999681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c096ed-1fcc-4b62-872e-8e2c2e124d02 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:33.999786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c096ed-1fcc-4b62-872e-8e2c2e124d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:34.017230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-83c096ed-1fcc-4b62-872e-8e2c2e124d02 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:53:34.018419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-83c096ed-1fcc-4b62-872e-8e2c2e124d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:53:34.026311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 736/1492] 10.222.0.121 () {58 vars in 1028 bytes} [Thu May 26 17:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:53:34.066400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7660c-4e87-4dd2-93ce-6a78f38aabbc None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:34.077393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7660c-4e87-4dd2-93ce-6a78f38aabbc None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:34.080953 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26ee58-c723-4026-8d89-4d37ebd29b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:34.081597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26ee58-c723-4026-8d89-4d37ebd29b1c service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:34.081944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26ee58-c723-4026-8d89-4d37ebd29b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:34.082222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26ee58-c723-4026-8d89-4d37ebd29b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:34.091727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26ee58-c723-4026-8d89-4d37ebd29b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:53:34.093184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26ee58-c723-4026-8d89-4d37ebd29b1c service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:53:34.103917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 757/1493] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:53:44.012028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=33 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:53:44.099363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=33 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:55:46.425097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c23b1-33ac-4494-ba95-e776d21804a7 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:46.425685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c23b1-33ac-4494-ba95-e776d21804a7 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:46.425685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c23b1-33ac-4494-ba95-e776d21804a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:46.444080 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-175c23b1-33ac-4494-ba95-e776d21804a7 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:55:46.445264 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-175c23b1-33ac-4494-ba95-e776d21804a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:46.453237 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 737/1494] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:55:46.462607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dbf09-c947-4039-9667-80689b63c6a5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:46.467600 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dbf09-c947-4039-9667-80689b63c6a5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:46.467600 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dbf09-c947-4039-9667-80689b63c6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:46.482911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-720dbf09-c947-4039-9667-80689b63c6a5 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:55:46.484106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-720dbf09-c947-4039-9667-80689b63c6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:46.491900 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 758/1495] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:55:46.497804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c23b1-33ac-4494-ba95-e776d21804a7 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:55:46.507819 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c23b1-33ac-4494-ba95-e776d21804a7 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:55:46.512488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36a141-1772-4d76-ba50-3804083d5fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:55:46.513114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36a141-1772-4d76-ba50-3804083d5fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:46.513391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36a141-1772-4d76-ba50-3804083d5fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:46.513660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36a141-1772-4d76-ba50-3804083d5fc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:46.514412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36a141-1772-4d76-ba50-3804083d5fc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:55:46.514705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36a141-1772-4d76-ba50-3804083d5fc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:55:46.515728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: WARNING py.warnings [None req-7a36a141-1772-4d76-ba50-3804083d5fc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 17:55:46.515728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: warnings.warn(msg) May 26 17:55:46.515728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: May 26 17:55:46.516595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36a141-1772-4d76-ba50-3804083d5fc3 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:55:46.529159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 738/1496] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 17:55:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:55:46.538295 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dbf09-c947-4039-9667-80689b63c6a5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:55:46.548456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dbf09-c947-4039-9667-80689b63c6a5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:55:46.553395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f64dd5-3a03-4269-900b-0584eae86680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:55:46.554072 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f64dd5-3a03-4269-900b-0584eae86680 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:46.554348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f64dd5-3a03-4269-900b-0584eae86680 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:46.554618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f64dd5-3a03-4269-900b-0584eae86680 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:46.555347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f64dd5-3a03-4269-900b-0584eae86680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:55:46.557152 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-b4f64dd5-3a03-4269-900b-0584eae86680 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 17:55:46.557152 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: warnings.warn(msg) May 26 17:55:46.557152 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: May 26 17:55:46.559037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f64dd5-3a03-4269-900b-0584eae86680 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:55:46.655534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 759/1497] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:55:46 2022] POST /identity/v3/projects => generated 376 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:55:46.659981 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36a141-1772-4d76-ba50-3804083d5fc3 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:55:46.670651 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36a141-1772-4d76-ba50-3804083d5fc3 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:55:46.726369 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd036ce8-649c-40bc-afc9-5625c7e4258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:55:46.727070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd036ce8-649c-40bc-afc9-5625c7e4258b admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:46.727347 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd036ce8-649c-40bc-afc9-5625c7e4258b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:46.727616 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd036ce8-649c-40bc-afc9-5625c7e4258b admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:46.728299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd036ce8-649c-40bc-afc9-5625c7e4258b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:55:46.729300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: WARNING py.warnings [None req-dd036ce8-649c-40bc-afc9-5625c7e4258b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 17:55:46.729300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: warnings.warn(msg) May 26 17:55:46.729300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: May 26 17:55:46.731132 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd036ce8-649c-40bc-afc9-5625c7e4258b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:55:46.807946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 739/1498] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:55:46 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:55:46.812602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f64dd5-3a03-4269-900b-0584eae86680 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:55:46.823601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f64dd5-3a03-4269-900b-0584eae86680 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:55:46.828142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4b7e4-ecfa-4101-bd27-6f75521aea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:55:46.828839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4b7e4-ecfa-4101-bd27-6f75521aea4e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:46.829115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4b7e4-ecfa-4101-bd27-6f75521aea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:46.829392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4b7e4-ecfa-4101-bd27-6f75521aea4e admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:46.830107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4b7e4-ecfa-4101-bd27-6f75521aea4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:55:46.831147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-cef4b7e4-ecfa-4101-bd27-6f75521aea4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 17:55:46.831147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: warnings.warn(msg) May 26 17:55:46.831147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: May 26 17:55:46.832104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4b7e4-ecfa-4101-bd27-6f75521aea4e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:55:46.845683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 760/1499] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:55:46 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:55:46.850028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd036ce8-649c-40bc-afc9-5625c7e4258b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:55:46.860845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd036ce8-649c-40bc-afc9-5625c7e4258b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:55:46.869493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed123b-833d-4557-8043-02243ce02660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:55:46.870327 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed123b-833d-4557-8043-02243ce02660 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:46.870812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed123b-833d-4557-8043-02243ce02660 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:46.871227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed123b-833d-4557-8043-02243ce02660 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:46.871946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed123b-833d-4557-8043-02243ce02660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:55:46.872957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: WARNING py.warnings [None req-baed123b-833d-4557-8043-02243ce02660 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 17:55:46.872957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: warnings.warn(msg) May 26 17:55:46.872957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: May 26 17:55:46.873922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed123b-833d-4557-8043-02243ce02660 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:55:46.888392 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 740/1500] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:55:46 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:55:46.892966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4b7e4-ecfa-4101-bd27-6f75521aea4e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:55:46.903761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4b7e4-ecfa-4101-bd27-6f75521aea4e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:55:46.908292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba963a-4195-493c-8a97-fcda78ea9990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:55:46.909444 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba963a-4195-493c-8a97-fcda78ea9990 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:46.909691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba963a-4195-493c-8a97-fcda78ea9990 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:46.909807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba963a-4195-493c-8a97-fcda78ea9990 admin admin] PATH_INFO: `/v3/projects/58c3949623d24467a2884c9d234c96d0/users/a52dc03b56a14fcaac9c298d73eb6c31/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:46.925371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba963a-4195-493c-8a97-fcda78ea9990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c3949623d24467a2884c9d234c96d0, user_id=a52dc03b56a14fcaac9c298d73eb6c31, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:55:46.926484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-13ba963a-4195-493c-8a97-fcda78ea9990 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 17:55:46.926484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: warnings.warn(msg) May 26 17:55:46.926484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: May 26 17:55:46.928687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba963a-4195-493c-8a97-fcda78ea9990 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:55:46.978654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 761/1501] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:55:46 2022] PUT /identity/v3/projects/58c3949623d24467a2884c9d234c96d0/users/a52dc03b56a14fcaac9c298d73eb6c31/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:55:46.991636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5855a0-fe15-4a83-b8da-17af96ebbec8 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:46.991924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5855a0-fe15-4a83-b8da-17af96ebbec8 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:46.992205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5855a0-fe15-4a83-b8da-17af96ebbec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:47.019345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-0e5855a0-fe15-4a83-b8da-17af96ebbec8 None None] MFA Rules not processed for user `a52dc03b56a14fcaac9c298d73eb6c31`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:55:47.020561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0e5855a0-fe15-4a83-b8da-17af96ebbec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:47.085655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 741/1502] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:55:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:55:47.099965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4de476-6f56-4269-907c-88459177a2ae None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:47.100255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4de476-6f56-4269-907c-88459177a2ae None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:47.100532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4de476-6f56-4269-907c-88459177a2ae None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:47.101888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 762/1503] 10.222.0.121 () {54 vars in 968 bytes} [Thu May 26 17:55:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:55:47.107140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913262f-231e-4ef3-8602-66fc5c11215f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:47.107399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913262f-231e-4ef3-8602-66fc5c11215f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:47.107676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913262f-231e-4ef3-8602-66fc5c11215f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:47.126146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d913262f-231e-4ef3-8602-66fc5c11215f None None] MFA Rules not processed for user `25d85d02abed40199dd2515ecf84b7fb`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:55:47.127391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d913262f-231e-4ef3-8602-66fc5c11215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:47.175520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:55:47.183372 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 742/1504] 10.222.0.121 () {58 vars in 1065 bytes} [Thu May 26 17:55:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:55:47.300128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4de476-6f56-4269-907c-88459177a2ae None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:55:47.311123 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4de476-6f56-4269-907c-88459177a2ae None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:55:47.314648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a21eef-72a5-419b-9ff7-ef9a84c0d3e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:55:47.315328 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a21eef-72a5-419b-9ff7-ef9a84c0d3e1 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:47.315605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a21eef-72a5-419b-9ff7-ef9a84c0d3e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:47.315878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a21eef-72a5-419b-9ff7-ef9a84c0d3e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:47.325224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a21eef-72a5-419b-9ff7-ef9a84c0d3e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:55:47.326658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a21eef-72a5-419b-9ff7-ef9a84c0d3e1 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:55:47.335814 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 763/1505] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:55:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:55:47.379246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e32c75-81ee-4505-97fb-0c0f223d69fa None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:47.379557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e32c75-81ee-4505-97fb-0c0f223d69fa None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:47.379832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e32c75-81ee-4505-97fb-0c0f223d69fa None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:47.381119 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 743/1506] 10.222.0.121 () {52 vars in 834 bytes} [Thu May 26 17:55:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:55:47.384774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a21eef-72a5-419b-9ff7-ef9a84c0d3e1 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:55:47.448556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a21eef-72a5-419b-9ff7-ef9a84c0d3e1 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:55:47.453260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:55:47.454143 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:47.454424 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:47.454696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] PATH_INFO: `/v3/users/a52dc03b56a14fcaac9c298d73eb6c31` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:47.456010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] RBAC: Authorizing `identity:delete_user(user_id=a52dc03b56a14fcaac9c298d73eb6c31)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:55:47.463297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 17:55:47.463297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: warnings.warn(msg) May 26 17:55:47.463297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: May 26 17:55:47.463297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:55:47.667557 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a52dc03b56a14fcaac9c298d73eb6c31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:55:47.686384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a52dc03b56a14fcaac9c298d73eb6c31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:55:47.687265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a52dc03b56a14fcaac9c298d73eb6c31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:55:47.707125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a52dc03b56a14fcaac9c298d73eb6c31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:55:47.711100 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:55:47.845552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a52dc03b56a14fcaac9c298d73eb6c31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:55:47.852104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 764/1507] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 17:55:47 2022] DELETE /identity/v3/users/a52dc03b56a14fcaac9c298d73eb6c31 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:55:47.856654 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecdbdb-c60b-48d1-bf98-370382d16d90 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:47.857173 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecdbdb-c60b-48d1-bf98-370382d16d90 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:47.857173 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecdbdb-c60b-48d1-bf98-370382d16d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:47.880039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5fecdbdb-c60b-48d1-bf98-370382d16d90 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:55:47.881236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5fecdbdb-c60b-48d1-bf98-370382d16d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:47.941620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 744/1508] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:55:47.953427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:55:47.954861 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:48.012661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd60464-9d60-48cb-a891-cf33302504af admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:55:48.016178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d66a8-999f-4acd-b8c4-13e46000f86e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:55:48.016828 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d66a8-999f-4acd-b8c4-13e46000f86e service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:48.017109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d66a8-999f-4acd-b8c4-13e46000f86e service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:48.017383 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d66a8-999f-4acd-b8c4-13e46000f86e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:48.027601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d66a8-999f-4acd-b8c4-13e46000f86e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:55:48.029036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d66a8-999f-4acd-b8c4-13e46000f86e service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:55:48.038273 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 765/1509] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:55:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:55:48.524205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecdbdb-c60b-48d1-bf98-370382d16d90 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:55:48.525908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5fecdbdb-c60b-48d1-bf98-370382d16d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:48.538219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecdbdb-c60b-48d1-bf98-370382d16d90 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:55:48.545779 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:55:48.546656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:48.546961 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:48.547233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] PATH_INFO: `/v3/projects/58c3949623d24467a2884c9d234c96d0` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:48.548448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c3949623d24467a2884c9d234c96d0)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:55:48.549455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: WARNING py.warnings [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 17:55:48.549455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: warnings.warn(msg) May 26 17:55:48.549455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: May 26 17:55:48.550640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:55:48.670229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c3949623d24467a2884c9d234c96d0', 'request_id': 'req-8848d15e-2e64-4ea2-b731-686e900fa537'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:55:48.750629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c3949623d24467a2884c9d234c96d0', 'request_id': 'req-8848d15e-2e64-4ea2-b731-686e900fa537'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:55:48.766558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c3949623d24467a2884c9d234c96d0', 'request_id': 'req-8848d15e-2e64-4ea2-b731-686e900fa537'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:55:48.780162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] The token cache is being invalidate because project 58c3949623d24467a2884c9d234c96d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 17:55:48.780455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:55:48.781137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-8848d15e-2e64-4ea2-b731-686e900fa537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:55:48.785760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 745/1510] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 17:55:48 2022] DELETE /identity/v3/projects/58c3949623d24467a2884c9d234c96d0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:55:58.034232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:55:58.776406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:57:30.787641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90a31c-ac81-428c-997b-dfd116345e90 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:30.788255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90a31c-ac81-428c-997b-dfd116345e90 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:30.788255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90a31c-ac81-428c-997b-dfd116345e90 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:30.789631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:57:30 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:57:30.794791 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910e9b7-5852-49b2-ad3e-b7483607a7cc None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:30.795294 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910e9b7-5852-49b2-ad3e-b7483607a7cc None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:30.795408 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910e9b7-5852-49b2-ad3e-b7483607a7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:30.815654 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-4910e9b7-5852-49b2-ad3e-b7483607a7cc None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:30.816956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4910e9b7-5852-49b2-ad3e-b7483607a7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:30.885079 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:31.058853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90a31c-ac81-428c-997b-dfd116345e90 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:31.061030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3e90a31c-ac81-428c-997b-dfd116345e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:31.142068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90a31c-ac81-428c-997b-dfd116345e90 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:31.148582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d3415-5049-4c07-98a7-d314d8535447 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:31.149260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d3415-5049-4c07-98a7-d314d8535447 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:31.149543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d3415-5049-4c07-98a7-d314d8535447 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:31.149821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d3415-5049-4c07-98a7-d314d8535447 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:31.158582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d3415-5049-4c07-98a7-d314d8535447 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:31.160242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d3415-5049-4c07-98a7-d314d8535447 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:31.173908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 767/1513] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:57:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:32.080042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb32cf-5282-4da5-a3f6-eec6e313058a None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:32.080621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb32cf-5282-4da5-a3f6-eec6e313058a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:32.080621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb32cf-5282-4da5-a3f6-eec6e313058a None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:32.081946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:57:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:57:32.087251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ff7fc-cfdf-4693-885b-f20b7ba8a34e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:32.087847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ff7fc-cfdf-4693-885b-f20b7ba8a34e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:32.087847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ff7fc-cfdf-4693-885b-f20b7ba8a34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:32.106574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a01ff7fc-cfdf-4693-885b-f20b7ba8a34e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:32.107831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a01ff7fc-cfdf-4693-885b-f20b7ba8a34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:32.116236 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:32.275262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb32cf-5282-4da5-a3f6-eec6e313058a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:32.290207 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb32cf-5282-4da5-a3f6-eec6e313058a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:32.296498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12c71b-ff32-4e6f-b303-e8e94d560663 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:32.297151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12c71b-ff32-4e6f-b303-e8e94d560663 service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:32.297430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12c71b-ff32-4e6f-b303-e8e94d560663 service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:32.297701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12c71b-ff32-4e6f-b303-e8e94d560663 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:32.307285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12c71b-ff32-4e6f-b303-e8e94d560663 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:32.308875 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12c71b-ff32-4e6f-b303-e8e94d560663 service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:32.320202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 748/1516] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:57:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:32.476369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b097c24-abbc-4edc-8e01-85f6a7dfa5b2 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:32.476689 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b097c24-abbc-4edc-8e01-85f6a7dfa5b2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:32.476913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b097c24-abbc-4edc-8e01-85f6a7dfa5b2 None None] PATH_INFO: `/v3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:32.478234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 769/1517] 10.222.0.129 () {54 vars in 930 bytes} [Thu May 26 17:57:32 2022] GET /identity/v3 => generated 256 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:57:32.484542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b705d8-f88b-4db8-b1f8-bc24ec2fd15a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:32.484817 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b705d8-f88b-4db8-b1f8-bc24ec2fd15a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:32.485097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b705d8-f88b-4db8-b1f8-bc24ec2fd15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:32.505350 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-64b705d8-f88b-4db8-b1f8-bc24ec2fd15a None None] MFA Rules not processed for user `7847d2c119474d49bef6029d51dbefdb`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:32.506647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-64b705d8-f88b-4db8-b1f8-bc24ec2fd15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:32.571742 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 749/1518] 10.222.0.129 () {58 vars in 1020 bytes} [Thu May 26 17:57:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:36.869116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cc517-2861-40da-9476-0ac48f8e7481 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:36.869685 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cc517-2861-40da-9476-0ac48f8e7481 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:36.869685 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cc517-2861-40da-9476-0ac48f8e7481 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:36.871065 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 770/1519] 10.222.0.121 () {54 vars in 965 bytes} [Thu May 26 17:57:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:57:36.876281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56757a57-5801-4e58-bece-4657c0ac5ec4 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:36.876724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56757a57-5801-4e58-bece-4657c0ac5ec4 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:36.876826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56757a57-5801-4e58-bece-4657c0ac5ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:36.896841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-56757a57-5801-4e58-bece-4657c0ac5ec4 None None] MFA Rules not processed for user `7847d2c119474d49bef6029d51dbefdb`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:36.898101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-56757a57-5801-4e58-bece-4657c0ac5ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:36.907186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 750/1520] 10.222.0.121 () {58 vars in 1062 bytes} [Thu May 26 17:57:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:36.914891 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cc517-2861-40da-9476-0ac48f8e7481 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:36.928882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cc517-2861-40da-9476-0ac48f8e7481 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:36.933742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163f2ea-ac41-42a4-90e5-95b3b1ab0d14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:36.934373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163f2ea-ac41-42a4-90e5-95b3b1ab0d14 service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:36.934984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163f2ea-ac41-42a4-90e5-95b3b1ab0d14 service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:36.935178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163f2ea-ac41-42a4-90e5-95b3b1ab0d14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:36.943986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163f2ea-ac41-42a4-90e5-95b3b1ab0d14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:36.945572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163f2ea-ac41-42a4-90e5-95b3b1ab0d14 service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:36.955554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 771/1521] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:57:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:43.629305 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3d393-ff88-4fee-9fe3-7f8d56ed84dd None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:43.629878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3d393-ff88-4fee-9fe3-7f8d56ed84dd None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:43.629878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3d393-ff88-4fee-9fe3-7f8d56ed84dd None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:43.631290 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:57:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:57:43.636265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c5ddc-a445-4ad0-ac89-ce6abcb04677 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:43.636733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c5ddc-a445-4ad0-ac89-ce6abcb04677 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:43.636848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c5ddc-a445-4ad0-ac89-ce6abcb04677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:43.656108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9e1c5ddc-a445-4ad0-ac89-ce6abcb04677 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:43.657322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9e1c5ddc-a445-4ad0-ac89-ce6abcb04677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:43.665253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:43.822446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3d393-ff88-4fee-9fe3-7f8d56ed84dd None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:43.840421 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3d393-ff88-4fee-9fe3-7f8d56ed84dd None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:43.847768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc7c8d-2345-440f-aaf5-538a8195aca0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:43.848442 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc7c8d-2345-440f-aaf5-538a8195aca0 service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:43.848747 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc7c8d-2345-440f-aaf5-538a8195aca0 service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:43.849006 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc7c8d-2345-440f-aaf5-538a8195aca0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:43.858349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc7c8d-2345-440f-aaf5-538a8195aca0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:43.860860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc7c8d-2345-440f-aaf5-538a8195aca0 service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:43.871827 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 752/1524] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:57:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:44.792542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0356f5-57a7-47af-9a22-460fc6d7c38c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:44.793105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0356f5-57a7-47af-9a22-460fc6d7c38c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:44.793105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0356f5-57a7-47af-9a22-460fc6d7c38c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:44.794578 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:57:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:57:44.799643 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fb5d6-187a-4313-91b6-6bbd98ddbc32 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:44.800143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fb5d6-187a-4313-91b6-6bbd98ddbc32 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:44.800753 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fb5d6-187a-4313-91b6-6bbd98ddbc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:44.821677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3a4fb5d6-187a-4313-91b6-6bbd98ddbc32 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:44.822901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3a4fb5d6-187a-4313-91b6-6bbd98ddbc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:44.830742 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:44.986051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0356f5-57a7-47af-9a22-460fc6d7c38c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:45.000836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0356f5-57a7-47af-9a22-460fc6d7c38c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:45.006468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be75e15-0c8c-4435-af67-8fdd05ff1677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:45.007187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be75e15-0c8c-4435-af67-8fdd05ff1677 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:45.007427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be75e15-0c8c-4435-af67-8fdd05ff1677 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:45.007696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be75e15-0c8c-4435-af67-8fdd05ff1677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:45.016419 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be75e15-0c8c-4435-af67-8fdd05ff1677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:45.018012 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be75e15-0c8c-4435-af67-8fdd05ff1677 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:45.029504 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 774/1527] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:49.277703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605eabb-6ba6-4eb5-9879-9ac440d0d935 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:49.278284 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605eabb-6ba6-4eb5-9879-9ac440d0d935 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:49.278284 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605eabb-6ba6-4eb5-9879-9ac440d0d935 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:49.279673 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:57:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:57:49.285054 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d871db7-5092-4ac2-b0a7-c3bef8649195 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:49.285513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d871db7-5092-4ac2-b0a7-c3bef8649195 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:49.285726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d871db7-5092-4ac2-b0a7-c3bef8649195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:49.304859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2d871db7-5092-4ac2-b0a7-c3bef8649195 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:49.306054 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2d871db7-5092-4ac2-b0a7-c3bef8649195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:49.314216 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:49.351476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605eabb-6ba6-4eb5-9879-9ac440d0d935 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:49.353248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c605eabb-6ba6-4eb5-9879-9ac440d0d935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:49.420296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605eabb-6ba6-4eb5-9879-9ac440d0d935 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:49.423832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c6bc5-0b32-4afc-b02f-e0c4eedc4678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:49.424486 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c6bc5-0b32-4afc-b02f-e0c4eedc4678 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:49.424773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c6bc5-0b32-4afc-b02f-e0c4eedc4678 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:49.425935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c6bc5-0b32-4afc-b02f-e0c4eedc4678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:49.435780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c6bc5-0b32-4afc-b02f-e0c4eedc4678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:49.437194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c6bc5-0b32-4afc-b02f-e0c4eedc4678 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:49.442718 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:57:49.447686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 755/1530] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:51.700390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b73d49-0a30-4a8a-bc15-26605f491b20 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:51.700989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b73d49-0a30-4a8a-bc15-26605f491b20 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:51.700989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b73d49-0a30-4a8a-bc15-26605f491b20 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:51.702325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:57:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:57:51.707723 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff0c0d-7cad-4006-b257-bb13f7d61f99 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:51.708157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff0c0d-7cad-4006-b257-bb13f7d61f99 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:51.708267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff0c0d-7cad-4006-b257-bb13f7d61f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:51.726599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-20ff0c0d-7cad-4006-b257-bb13f7d61f99 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:51.727831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-20ff0c0d-7cad-4006-b257-bb13f7d61f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:51.735493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:51.772084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b73d49-0a30-4a8a-bc15-26605f491b20 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:51.783113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b73d49-0a30-4a8a-bc15-26605f491b20 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:51.786983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982eb94-626b-4c7d-b9c9-f0e2e666da85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:51.787705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982eb94-626b-4c7d-b9c9-f0e2e666da85 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:51.787907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982eb94-626b-4c7d-b9c9-f0e2e666da85 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:51.788181 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982eb94-626b-4c7d-b9c9-f0e2e666da85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:51.799483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982eb94-626b-4c7d-b9c9-f0e2e666da85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:51.801872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982eb94-626b-4c7d-b9c9-f0e2e666da85 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:51.812080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 777/1533] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:57:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:55.623795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d0135-466d-4727-9afb-de516933ce7e None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:55.624382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d0135-466d-4727-9afb-de516933ce7e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:55.624382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d0135-466d-4727-9afb-de516933ce7e None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:55.625723 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:57:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:57:55.631717 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20e665-38e8-4b60-b82b-0980f001680c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:55.632142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20e665-38e8-4b60-b82b-0980f001680c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:55.632251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20e665-38e8-4b60-b82b-0980f001680c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:55.651473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7e20e665-38e8-4b60-b82b-0980f001680c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:55.652712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7e20e665-38e8-4b60-b82b-0980f001680c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:55.660907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:55.934678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d0135-466d-4727-9afb-de516933ce7e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:55.936623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ee7d0135-466d-4727-9afb-de516933ce7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:56.008610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d0135-466d-4727-9afb-de516933ce7e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:56.012266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3c3b2-a208-4852-8110-d35942c1ab99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:56.013063 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c3b2-a208-4852-8110-d35942c1ab99 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:56.013349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c3b2-a208-4852-8110-d35942c1ab99 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:56.013645 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c3b2-a208-4852-8110-d35942c1ab99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:56.025429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3c3b2-a208-4852-8110-d35942c1ab99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:56.026893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3c3b2-a208-4852-8110-d35942c1ab99 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:56.036977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 758/1536] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:58.298152 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb0a18-b282-4b0a-9eaa-64c7cb30cdc5 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:58.298714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb0a18-b282-4b0a-9eaa-64c7cb30cdc5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:58.298714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb0a18-b282-4b0a-9eaa-64c7cb30cdc5 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:58.301121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:57:58 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:57:58.306735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec65323-7b82-4398-aa6d-53a6b17cf8fb None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:58.307296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec65323-7b82-4398-aa6d-53a6b17cf8fb None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:58.307424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec65323-7b82-4398-aa6d-53a6b17cf8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:58.326020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-0ec65323-7b82-4398-aa6d-53a6b17cf8fb None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:58.327273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0ec65323-7b82-4398-aa6d-53a6b17cf8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:58.335839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:58.615423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb0a18-b282-4b0a-9eaa-64c7cb30cdc5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:58.617089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-68cb0a18-b282-4b0a-9eaa-64c7cb30cdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:58.624754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=65 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:57:58.632953 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb0a18-b282-4b0a-9eaa-64c7cb30cdc5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:58.636778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde19ce-c07f-47b4-89d5-f8b5c1ce0951 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:58.641385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde19ce-c07f-47b4-89d5-f8b5c1ce0951 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:58.641385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde19ce-c07f-47b4-89d5-f8b5c1ce0951 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:58.641385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde19ce-c07f-47b4-89d5-f8b5c1ce0951 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:58.647317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde19ce-c07f-47b4-89d5-f8b5c1ce0951 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:58.649006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde19ce-c07f-47b4-89d5-f8b5c1ce0951 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:58.658457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 780/1539] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:58:08.320372 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=32 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:58:08.662448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:30.745297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a00fb-c52d-4ca6-86d0-9ca51b7bbaef None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:30.745949 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a00fb-c52d-4ca6-86d0-9ca51b7bbaef None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:30.745949 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a00fb-c52d-4ca6-86d0-9ca51b7bbaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:30.776127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-886a00fb-c52d-4ca6-86d0-9ca51b7bbaef None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:30.777768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-886a00fb-c52d-4ca6-86d0-9ca51b7bbaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:30.792538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 760/1540] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:30.811655 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f476d-b26a-468d-a699-d3c0bd0f4ee2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:30.812585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f476d-b26a-468d-a699-d3c0bd0f4ee2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:30.813029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f476d-b26a-468d-a699-d3c0bd0f4ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:30.837729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-da0f476d-b26a-468d-a699-d3c0bd0f4ee2 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:30.839585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-da0f476d-b26a-468d-a699-d3c0bd0f4ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:30.858619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 781/1541] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:30.864015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a00fb-c52d-4ca6-86d0-9ca51b7bbaef None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:30.877861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a00fb-c52d-4ca6-86d0-9ca51b7bbaef None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:30.888906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb023d-7fa9-4ff9-b134-e6f6d7f688f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:30.890888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb023d-7fa9-4ff9-b134-e6f6d7f688f3 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:30.891337 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb023d-7fa9-4ff9-b134-e6f6d7f688f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:30.891767 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb023d-7fa9-4ff9-b134-e6f6d7f688f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:30.892900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb023d-7fa9-4ff9-b134-e6f6d7f688f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:59:30.893382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb023d-7fa9-4ff9-b134-e6f6d7f688f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:30.895186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb023d-7fa9-4ff9-b134-e6f6d7f688f3 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:30.914612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 761/1542] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 17:59:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:59:30.923629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f476d-b26a-468d-a699-d3c0bd0f4ee2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:30.933996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f476d-b26a-468d-a699-d3c0bd0f4ee2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:30.943525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baaab7-7c40-4119-b8bf-662be15d56a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:30.944248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baaab7-7c40-4119-b8bf-662be15d56a8 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:30.944525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baaab7-7c40-4119-b8bf-662be15d56a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:30.944804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baaab7-7c40-4119-b8bf-662be15d56a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:30.945535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baaab7-7c40-4119-b8bf-662be15d56a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:30.947885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baaab7-7c40-4119-b8bf-662be15d56a8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:30.988358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c98db-74bc-427c-9a44-4935cacb0a04 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.000822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c98db-74bc-427c-9a44-4935cacb0a04 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.000822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c98db-74bc-427c-9a44-4935cacb0a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.021244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-8a2c98db-74bc-427c-9a44-4935cacb0a04 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.023444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-8a2c98db-74bc-427c-9a44-4935cacb0a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.035332 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 762/1543] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:59:31.046348 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382ebe7-e70c-454c-98b5-e4e09fecd18a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.047234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382ebe7-e70c-454c-98b5-e4e09fecd18a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.047718 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382ebe7-e70c-454c-98b5-e4e09fecd18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.069014 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 782/1544] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:30 2022] POST /identity/v3/projects => generated 392 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:31.075246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baaab7-7c40-4119-b8bf-662be15d56a8 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:31.080117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d382ebe7-e70c-454c-98b5-e4e09fecd18a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.081833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d382ebe7-e70c-454c-98b5-e4e09fecd18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.182655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 763/1545] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.188892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382ebe7-e70c-454c-98b5-e4e09fecd18a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:31.215709 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382ebe7-e70c-454c-98b5-e4e09fecd18a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:31.227715 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baaab7-7c40-4119-b8bf-662be15d56a8 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:31.236205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5462e-d05b-424a-9b16-cd926a4e5709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:31.237281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5462e-d05b-424a-9b16-cd926a4e5709 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.237775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5462e-d05b-424a-9b16-cd926a4e5709 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.238255 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5462e-d05b-424a-9b16-cd926a4e5709 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.239664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5462e-d05b-424a-9b16-cd926a4e5709 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:59:31.240429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5462e-d05b-424a-9b16-cd926a4e5709 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:31.242467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5462e-d05b-424a-9b16-cd926a4e5709 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:31.251662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cb0fc-bcf3-493b-ab7d-70c441ba0129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:31.251662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cb0fc-bcf3-493b-ab7d-70c441ba0129 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.251662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cb0fc-bcf3-493b-ab7d-70c441ba0129 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.251662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cb0fc-bcf3-493b-ab7d-70c441ba0129 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.252051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958cb0fc-bcf3-493b-ab7d-70c441ba0129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:31.252051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-958cb0fc-bcf3-493b-ab7d-70c441ba0129 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 17:59:31.252051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: warnings.warn(msg) May 26 17:59:31.252051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: May 26 17:59:31.252051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958cb0fc-bcf3-493b-ab7d-70c441ba0129 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:31.264778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 764/1546] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 17:59:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:59:31.266804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2193232-ef2f-41bd-aed8-88302aa65758 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.271828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2193232-ef2f-41bd-aed8-88302aa65758 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.272378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2193232-ef2f-41bd-aed8-88302aa65758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.306628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f2193232-ef2f-41bd-aed8-88302aa65758 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.308311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f2193232-ef2f-41bd-aed8-88302aa65758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.322784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 765/1547] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.324566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2193232-ef2f-41bd-aed8-88302aa65758 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:31.342086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2193232-ef2f-41bd-aed8-88302aa65758 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:31.350143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72192-015b-494f-9328-573e06830343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:31.351610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72192-015b-494f-9328-573e06830343 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.352503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72192-015b-494f-9328-573e06830343 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.353025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72192-015b-494f-9328-573e06830343 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.354030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d72192-015b-494f-9328-573e06830343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:31.356922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: WARNING py.warnings [None req-89d72192-015b-494f-9328-573e06830343 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 17:59:31.356922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: warnings.warn(msg) May 26 17:59:31.356922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: May 26 17:59:31.359650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d72192-015b-494f-9328-573e06830343 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:31.385473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 783/1548] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/users => generated 297 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:31.388816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997bec7-ee40-41fc-bc5a-02aca62cf8bb None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.388816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997bec7-ee40-41fc-bc5a-02aca62cf8bb None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.389878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997bec7-ee40-41fc-bc5a-02aca62cf8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.412594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7997bec7-ee40-41fc-bc5a-02aca62cf8bb None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.413879 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7997bec7-ee40-41fc-bc5a-02aca62cf8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.423258 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 784/1549] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.426938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8815b7-5066-4acb-b9dc-78993e8d8a2d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.426938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8815b7-5066-4acb-b9dc-78993e8d8a2d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.426938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8815b7-5066-4acb-b9dc-78993e8d8a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.446016 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6a8815b7-5066-4acb-b9dc-78993e8d8a2d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.447316 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6a8815b7-5066-4acb-b9dc-78993e8d8a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.457244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 785/1550] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.458651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937729e3-8d5f-4cd0-ba6b-b4f83d3cc48f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.458969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937729e3-8d5f-4cd0-ba6b-b4f83d3cc48f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.459249 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937729e3-8d5f-4cd0-ba6b-b4f83d3cc48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.482111 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 766/1551] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/projects => generated 404 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:31.484270 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72192-015b-494f-9328-573e06830343 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:31.485265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-937729e3-8d5f-4cd0-ba6b-b4f83d3cc48f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.486936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-937729e3-8d5f-4cd0-ba6b-b4f83d3cc48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.555128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 786/1552] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.561552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72192-015b-494f-9328-573e06830343 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:31.564327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b34606-731d-46ca-a32e-3b192d45f3f4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.564648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b34606-731d-46ca-a32e-3b192d45f3f4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.564928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b34606-731d-46ca-a32e-3b192d45f3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.566541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95af16-3460-4276-a949-38bd0d81b5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:31.567343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95af16-3460-4276-a949-38bd0d81b5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.567629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95af16-3460-4276-a949-38bd0d81b5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.567905 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95af16-3460-4276-a949-38bd0d81b5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.568653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95af16-3460-4276-a949-38bd0d81b5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:31.570239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95af16-3460-4276-a949-38bd0d81b5f7 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:31.582326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 767/1553] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:31 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:31.583655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4a204-af0c-40e0-90c2-9c34e5a0a44e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.583949 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4a204-af0c-40e0-90c2-9c34e5a0a44e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.584208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b9b34606-731d-46ca-a32e-3b192d45f3f4 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.584357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4a204-af0c-40e0-90c2-9c34e5a0a44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.585483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b9b34606-731d-46ca-a32e-3b192d45f3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.594539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 787/1554] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.598262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b34606-731d-46ca-a32e-3b192d45f3f4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:31.605467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b34606-731d-46ca-a32e-3b192d45f3f4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:31.608240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-efe4a204-af0c-40e0-90c2-9c34e5a0a44e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.609445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-efe4a204-af0c-40e0-90c2-9c34e5a0a44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.610620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc4773-7d2c-4961-b508-291b4dda6444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:31.611324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc4773-7d2c-4961-b508-291b4dda6444 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.611611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc4773-7d2c-4961-b508-291b4dda6444 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.611890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc4773-7d2c-4961-b508-291b4dda6444 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.612614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc4773-7d2c-4961-b508-291b4dda6444 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:59:31.612922 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc4773-7d2c-4961-b508-291b4dda6444 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:31.613933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-65cc4773-7d2c-4961-b508-291b4dda6444 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 17:59:31.613933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: warnings.warn(msg) May 26 17:59:31.613933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: May 26 17:59:31.615070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc4773-7d2c-4961-b508-291b4dda6444 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:31.618864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 768/1555] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.618864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4a204-af0c-40e0-90c2-9c34e5a0a44e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:31.628670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 788/1556] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 17:59:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:59:31.628976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4a204-af0c-40e0-90c2-9c34e5a0a44e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:31.630853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd574e-dde0-4068-897b-15f3215874f9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.631255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd574e-dde0-4068-897b-15f3215874f9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.631677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd574e-dde0-4068-897b-15f3215874f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.634230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c5465-e5a7-4e2f-81d5-77a326a75ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:31.634942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c5465-e5a7-4e2f-81d5-77a326a75ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.635285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c5465-e5a7-4e2f-81d5-77a326a75ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.635629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c5465-e5a7-4e2f-81d5-77a326a75ab8 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.636253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c5465-e5a7-4e2f-81d5-77a326a75ab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:31.638190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c5465-e5a7-4e2f-81d5-77a326a75ab8 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:31.651450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-02dd574e-dde0-4068-897b-15f3215874f9 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.652644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-02dd574e-dde0-4068-897b-15f3215874f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.660512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 789/1557] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.661951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270cdb7-3905-4d40-8100-3607f20c0bef None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.662232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270cdb7-3905-4d40-8100-3607f20c0bef None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.662606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270cdb7-3905-4d40-8100-3607f20c0bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.680351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e270cdb7-3905-4d40-8100-3607f20c0bef None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.681542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e270cdb7-3905-4d40-8100-3607f20c0bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.690265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 790/1558] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.691716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a322fd5-8405-48f0-afa1-1fdda2a53ded None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.692007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a322fd5-8405-48f0-afa1-1fdda2a53ded None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.692286 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a322fd5-8405-48f0-afa1-1fdda2a53ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.710904 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5a322fd5-8405-48f0-afa1-1fdda2a53ded None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.712116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5a322fd5-8405-48f0-afa1-1fdda2a53ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.720145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 791/1559] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.721048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a322fd5-8405-48f0-afa1-1fdda2a53ded None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:31.730311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a322fd5-8405-48f0-afa1-1fdda2a53ded None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:31.735114 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59c032-e37c-4a34-9c58-5d90b9d2c23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:31.735847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59c032-e37c-4a34-9c58-5d90b9d2c23e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.736132 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59c032-e37c-4a34-9c58-5d90b9d2c23e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.736400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59c032-e37c-4a34-9c58-5d90b9d2c23e admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.737139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59c032-e37c-4a34-9c58-5d90b9d2c23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:31.738882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59c032-e37c-4a34-9c58-5d90b9d2c23e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:31.741296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:31.749282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 769/1560] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/users => generated 303 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:31.749282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c5465-e5a7-4e2f-81d5-77a326a75ab8 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:31.753021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:31.755460 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 792/1561] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:31 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:31.757131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c5465-e5a7-4e2f-81d5-77a326a75ab8 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:31.757751 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100ba61-7758-4585-ae96-ba98decf10f1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.758241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100ba61-7758-4585-ae96-ba98decf10f1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.758720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100ba61-7758-4585-ae96-ba98decf10f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.764836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c848f-740f-44e3-97d4-d6700c5d66f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:31.765507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c848f-740f-44e3-97d4-d6700c5d66f1 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.765797 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c848f-740f-44e3-97d4-d6700c5d66f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.766074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c848f-740f-44e3-97d4-d6700c5d66f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.766856 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c848f-740f-44e3-97d4-d6700c5d66f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:59:31.767176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c848f-740f-44e3-97d4-d6700c5d66f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:31.768925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c848f-740f-44e3-97d4-d6700c5d66f1 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:31.781069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8100ba61-7758-4585-ae96-ba98decf10f1 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.782300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8100ba61-7758-4585-ae96-ba98decf10f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.783874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 770/1562] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 17:59:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:59:31.790543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c848f-740f-44e3-97d4-d6700c5d66f1 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:31.794091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 793/1563] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.794091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce29e1-6e4b-4085-bba3-dce40f90c20c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.794091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce29e1-6e4b-4085-bba3-dce40f90c20c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.794091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce29e1-6e4b-4085-bba3-dce40f90c20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.801366 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c848f-740f-44e3-97d4-d6700c5d66f1 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:31.806029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075764e8-0367-47b7-ba4b-7f16e87f2ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:31.806633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075764e8-0367-47b7-ba4b-7f16e87f2ffe admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.806962 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075764e8-0367-47b7-ba4b-7f16e87f2ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.812528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075764e8-0367-47b7-ba4b-7f16e87f2ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.812528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075764e8-0367-47b7-ba4b-7f16e87f2ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:31.812528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075764e8-0367-47b7-ba4b-7f16e87f2ffe admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:31.812825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-64ce29e1-6e4b-4085-bba3-dce40f90c20c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.813291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-64ce29e1-6e4b-4085-bba3-dce40f90c20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.821555 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 794/1564] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.823113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35672787-5bab-4c18-a19e-5c5d46871c86 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.823404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35672787-5bab-4c18-a19e-5c5d46871c86 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.823677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35672787-5bab-4c18-a19e-5c5d46871c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.842248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-35672787-5bab-4c18-a19e-5c5d46871c86 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:31.843536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-35672787-5bab-4c18-a19e-5c5d46871c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:31.859563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 795/1565] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:31.860438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35672787-5bab-4c18-a19e-5c5d46871c86 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:31.870403 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35672787-5bab-4c18-a19e-5c5d46871c86 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:31.876991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36256cc0-cce0-4e4e-9aeb-f40178fb78ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:31.877706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36256cc0-cce0-4e4e-9aeb-f40178fb78ec admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.877987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36256cc0-cce0-4e4e-9aeb-f40178fb78ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.878265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36256cc0-cce0-4e4e-9aeb-f40178fb78ec admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.879029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36256cc0-cce0-4e4e-9aeb-f40178fb78ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:59:31.879339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36256cc0-cce0-4e4e-9aeb-f40178fb78ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:31.880969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36256cc0-cce0-4e4e-9aeb-f40178fb78ec admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:31.895232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 796/1566] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 17:59:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:59:31.896303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36256cc0-cce0-4e4e-9aeb-f40178fb78ec admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:31.913047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 771/1567] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:31 2022] POST /identity/v3/projects => generated 404 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:31.913047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075764e8-0367-47b7-ba4b-7f16e87f2ffe admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:31.913379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36256cc0-cce0-4e4e-9aeb-f40178fb78ec admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:31.978320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075764e8-0367-47b7-ba4b-7f16e87f2ffe admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:31.985012 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc35255-c9eb-4ea8-a5ec-9bb265869ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:31.985533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec4749-c440-4df3-a727-120b59640d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:31.986032 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc35255-c9eb-4ea8-a5ec-9bb265869ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.986637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc35255-c9eb-4ea8-a5ec-9bb265869ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.986770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec4749-c440-4df3-a727-120b59640d52 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:31.986956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc35255-c9eb-4ea8-a5ec-9bb265869ab5 admin admin] PATH_INFO: `/v3/projects/3b0036437f8d4f2aaa19e707199916e3/users/dbee2da50ee6413db771071900703e29/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.987119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec4749-c440-4df3-a727-120b59640d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:31.987297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec4749-c440-4df3-a727-120b59640d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:31.988033 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec4749-c440-4df3-a727-120b59640d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:31.989803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec4749-c440-4df3-a727-120b59640d52 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.001917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 797/1568] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:31 2022] GET /identity/v3/roles => generated 1427 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:32.002925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec4749-c440-4df3-a727-120b59640d52 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.004385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc35255-c9eb-4ea8-a5ec-9bb265869ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0036437f8d4f2aaa19e707199916e3, user_id=dbee2da50ee6413db771071900703e29, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.005527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: WARNING py.warnings [None req-dbc35255-c9eb-4ea8-a5ec-9bb265869ab5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 17:59:32.005527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: warnings.warn(msg) May 26 17:59:32.005527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: May 26 17:59:32.007687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc35255-c9eb-4ea8-a5ec-9bb265869ab5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.014035 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec4749-c440-4df3-a727-120b59640d52 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.019369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e814302-de0c-4715-9244-057bfb16725a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.020079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e814302-de0c-4715-9244-057bfb16725a admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.020366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e814302-de0c-4715-9244-057bfb16725a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.020643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e814302-de0c-4715-9244-057bfb16725a admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.021410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e814302-de0c-4715-9244-057bfb16725a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.023665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e814302-de0c-4715-9244-057bfb16725a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.103695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 772/1569] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:31 2022] PUT /identity/v3/projects/3b0036437f8d4f2aaa19e707199916e3/users/dbee2da50ee6413db771071900703e29/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:32.104458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc35255-c9eb-4ea8-a5ec-9bb265869ab5 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.159856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 798/1570] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:32 2022] POST /identity/v3/projects => generated 384 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:32.161312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e814302-de0c-4715-9244-057bfb16725a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.176468 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc35255-c9eb-4ea8-a5ec-9bb265869ab5 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.228846 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e814302-de0c-4715-9244-057bfb16725a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.234119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f310f9-7637-4bff-8e41-75b45b7b5ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.234779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f310f9-7637-4bff-8e41-75b45b7b5ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.235087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f310f9-7637-4bff-8e41-75b45b7b5ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.235356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f310f9-7637-4bff-8e41-75b45b7b5ff5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.236071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f310f9-7637-4bff-8e41-75b45b7b5ff5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:59:32.236368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f310f9-7637-4bff-8e41-75b45b7b5ff5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.237144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2f8c7-0c0d-4e87-b45f-1e50d75c44b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.237822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2f8c7-0c0d-4e87-b45f-1e50d75c44b5 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.237987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f310f9-7637-4bff-8e41-75b45b7b5ff5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.238113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2f8c7-0c0d-4e87-b45f-1e50d75c44b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.238357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2f8c7-0c0d-4e87-b45f-1e50d75c44b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.239126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2f8c7-0c0d-4e87-b45f-1e50d75c44b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:59:32.239688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2f8c7-0c0d-4e87-b45f-1e50d75c44b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.241991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2f8c7-0c0d-4e87-b45f-1e50d75c44b5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.243804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:32.250870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 799/1571] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 17:59:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:59:32.255699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f310f9-7637-4bff-8e41-75b45b7b5ff5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.263863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 773/1572] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 17:59:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:59:32.263863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2f8c7-0c0d-4e87-b45f-1e50d75c44b5 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.267636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f310f9-7637-4bff-8e41-75b45b7b5ff5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.270976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2f8c7-0c0d-4e87-b45f-1e50d75c44b5 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.272582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007d00a-4a4f-4a13-9d49-05b7147603da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.273547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007d00a-4a4f-4a13-9d49-05b7147603da admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.273820 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007d00a-4a4f-4a13-9d49-05b7147603da admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.274088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007d00a-4a4f-4a13-9d49-05b7147603da admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.274848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007d00a-4a4f-4a13-9d49-05b7147603da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:59:32.275174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007d00a-4a4f-4a13-9d49-05b7147603da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.275910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23491ff5-80d6-4571-99f3-ae556770256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.276571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23491ff5-80d6-4571-99f3-ae556770256e admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.276794 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007d00a-4a4f-4a13-9d49-05b7147603da admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.276890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23491ff5-80d6-4571-99f3-ae556770256e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.277113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23491ff5-80d6-4571-99f3-ae556770256e admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.277844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23491ff5-80d6-4571-99f3-ae556770256e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.280074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23491ff5-80d6-4571-99f3-ae556770256e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.323353 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 800/1573] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 17:59:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:59:32.324196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007d00a-4a4f-4a13-9d49-05b7147603da admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.335550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007d00a-4a4f-4a13-9d49-05b7147603da admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.340367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aedf9-2bc8-447e-b787-64ae26db0b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.341084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aedf9-2bc8-447e-b787-64ae26db0b35 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.341358 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aedf9-2bc8-447e-b787-64ae26db0b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.341637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aedf9-2bc8-447e-b787-64ae26db0b35 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.342913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4aedf9-2bc8-447e-b787-64ae26db0b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.345029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4aedf9-2bc8-447e-b787-64ae26db0b35 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.367505 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 774/1574] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:32 2022] POST /identity/v3/projects => generated 384 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:32.368521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23491ff5-80d6-4571-99f3-ae556770256e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.431940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23491ff5-80d6-4571-99f3-ae556770256e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.436736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc419088-b916-4d75-95f2-8c5d88bb1fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.437406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc419088-b916-4d75-95f2-8c5d88bb1fbb admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.437683 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc419088-b916-4d75-95f2-8c5d88bb1fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.437953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc419088-b916-4d75-95f2-8c5d88bb1fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.438666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc419088-b916-4d75-95f2-8c5d88bb1fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.440290 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc419088-b916-4d75-95f2-8c5d88bb1fbb admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.451053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 775/1575] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:32 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:32.452577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fcb82-7ee1-4ff4-a05e-9777e4988d39 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.452890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fcb82-7ee1-4ff4-a05e-9777e4988d39 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.453171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fcb82-7ee1-4ff4-a05e-9777e4988d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.470431 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 801/1576] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:32 2022] POST /identity/v3/users => generated 303 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:32.471305 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aedf9-2bc8-447e-b787-64ae26db0b35 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.472508 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-453fcb82-7ee1-4ff4-a05e-9777e4988d39 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:32.473899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-453fcb82-7ee1-4ff4-a05e-9777e4988d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:32.476799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:32.482524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aedf9-2bc8-447e-b787-64ae26db0b35 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.482707 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 776/1577] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:32.483967 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fcb82-7ee1-4ff4-a05e-9777e4988d39 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.489718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f4ec2-86d5-4108-9461-bf1249b5048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.490545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f4ec2-86d5-4108-9461-bf1249b5048a admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.491091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f4ec2-86d5-4108-9461-bf1249b5048a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.491581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f4ec2-86d5-4108-9461-bf1249b5048a admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.492612 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f4ec2-86d5-4108-9461-bf1249b5048a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.494633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f4ec2-86d5-4108-9461-bf1249b5048a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.494777 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fcb82-7ee1-4ff4-a05e-9777e4988d39 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.501623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb43d40-f187-424e-9e87-29cf71555908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.502450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb43d40-f187-424e-9e87-29cf71555908 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.502736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb43d40-f187-424e-9e87-29cf71555908 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.503037 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb43d40-f187-424e-9e87-29cf71555908 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.503727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb43d40-f187-424e-9e87-29cf71555908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.506078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb43d40-f187-424e-9e87-29cf71555908 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.617713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 802/1578] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:32 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:32.618646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f4ec2-86d5-4108-9461-bf1249b5048a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.628853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f4ec2-86d5-4108-9461-bf1249b5048a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.634991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b41477-680d-4b3b-a6a8-c9b59a3ddb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.635662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b41477-680d-4b3b-a6a8-c9b59a3ddb1b admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.635940 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b41477-680d-4b3b-a6a8-c9b59a3ddb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.636212 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b41477-680d-4b3b-a6a8-c9b59a3ddb1b admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.640191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 777/1579] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:32 2022] POST /identity/v3/projects => generated 354 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:32.640377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b41477-680d-4b3b-a6a8-c9b59a3ddb1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.640493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb43d40-f187-424e-9e87-29cf71555908 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.640659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b41477-680d-4b3b-a6a8-c9b59a3ddb1b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.724442 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb43d40-f187-424e-9e87-29cf71555908 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.729156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7841c-9991-46d5-8dff-a8ed3e3acd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.729893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7841c-9991-46d5-8dff-a8ed3e3acd56 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.730175 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7841c-9991-46d5-8dff-a8ed3e3acd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.730453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7841c-9991-46d5-8dff-a8ed3e3acd56 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.731180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7841c-9991-46d5-8dff-a8ed3e3acd56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.733865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7841c-9991-46d5-8dff-a8ed3e3acd56 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.771125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 803/1580] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:32 2022] POST /identity/v3/projects => generated 388 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:32.771901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b41477-680d-4b3b-a6a8-c9b59a3ddb1b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.831891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 778/1581] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:32 2022] POST /identity/v3/projects => generated 376 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:32.832744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7841c-9991-46d5-8dff-a8ed3e3acd56 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:32.838854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b41477-680d-4b3b-a6a8-c9b59a3ddb1b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.863508 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:32.902314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4451ff7-8fb0-4511-95cc-f3f16a285521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.902949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4451ff7-8fb0-4511-95cc-f3f16a285521 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.903259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4451ff7-8fb0-4511-95cc-f3f16a285521 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.903540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4451ff7-8fb0-4511-95cc-f3f16a285521 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.904631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4451ff7-8fb0-4511-95cc-f3f16a285521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.907073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7841c-9991-46d5-8dff-a8ed3e3acd56 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:32.907347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4451ff7-8fb0-4511-95cc-f3f16a285521 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:32.911784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889b6cc-b44d-45dc-91b9-c612eff37149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:32.912681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889b6cc-b44d-45dc-91b9-c612eff37149 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:32.912965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889b6cc-b44d-45dc-91b9-c612eff37149 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:32.913236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889b6cc-b44d-45dc-91b9-c612eff37149 admin admin] PATH_INFO: `/v3/projects/18d58426facd4034a1e76fd7e52817b0/users/33c8564825094c4ca91bd2511fd3e425/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:32.929058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889b6cc-b44d-45dc-91b9-c612eff37149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18d58426facd4034a1e76fd7e52817b0, user_id=33c8564825094c4ca91bd2511fd3e425, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:32.931750 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889b6cc-b44d-45dc-91b9-c612eff37149 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.043865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 779/1582] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:32 2022] PUT /identity/v3/projects/18d58426facd4034a1e76fd7e52817b0/users/33c8564825094c4ca91bd2511fd3e425/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:33.044105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 804/1583] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:32 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:33.044880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4451ff7-8fb0-4511-95cc-f3f16a285521 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.045022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889b6cc-b44d-45dc-91b9-c612eff37149 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.106212 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4451ff7-8fb0-4511-95cc-f3f16a285521 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.118374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:33.124740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889b6cc-b44d-45dc-91b9-c612eff37149 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.129766 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208018b5-36cf-4f96-b8a9-58a19b414c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.132974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208018b5-36cf-4f96-b8a9-58a19b414c6c admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.132974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208018b5-36cf-4f96-b8a9-58a19b414c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.132974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208018b5-36cf-4f96-b8a9-58a19b414c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.132974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208018b5-36cf-4f96-b8a9-58a19b414c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.133409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208018b5-36cf-4f96-b8a9-58a19b414c6c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.147059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 780/1584] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:33.147059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208018b5-36cf-4f96-b8a9-58a19b414c6c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.157170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208018b5-36cf-4f96-b8a9-58a19b414c6c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.159678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e9e02-95b8-4168-b8c0-7e62eb374d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.160341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e9e02-95b8-4168-b8c0-7e62eb374d65 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.160621 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e9e02-95b8-4168-b8c0-7e62eb374d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.160893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e9e02-95b8-4168-b8c0-7e62eb374d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.162044 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6d029-6869-4d6f-a081-55723860b5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.162956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6d029-6869-4d6f-a081-55723860b5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.169480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6d029-6869-4d6f-a081-55723860b5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.169480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6d029-6869-4d6f-a081-55723860b5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.169480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6d029-6869-4d6f-a081-55723860b5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.169480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6d029-6869-4d6f-a081-55723860b5e9 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.171255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e9e02-95b8-4168-b8c0-7e62eb374d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.172733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e9e02-95b8-4168-b8c0-7e62eb374d65 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.176324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 781/1585] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:33.177139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6d029-6869-4d6f-a081-55723860b5e9 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.187658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 805/1586] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:33.189020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e9e02-95b8-4168-b8c0-7e62eb374d65 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.190444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6d029-6869-4d6f-a081-55723860b5e9 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.195425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208333e-e2dd-468b-b201-2d097e5bc702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.196699 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208333e-e2dd-468b-b201-2d097e5bc702 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.196699 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208333e-e2dd-468b-b201-2d097e5bc702 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.196699 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208333e-e2dd-468b-b201-2d097e5bc702 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.203058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208333e-e2dd-468b-b201-2d097e5bc702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.203058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208333e-e2dd-468b-b201-2d097e5bc702 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.203361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e9e02-95b8-4168-b8c0-7e62eb374d65 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.208551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c558c-c4c3-4342-92b4-9646e025ef95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.210187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c558c-c4c3-4342-92b4-9646e025ef95 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.210540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c558c-c4c3-4342-92b4-9646e025ef95 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.210959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c558c-c4c3-4342-92b4-9646e025ef95 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.211655 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c558c-c4c3-4342-92b4-9646e025ef95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.213588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c558c-c4c3-4342-92b4-9646e025ef95 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.305366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 806/1588] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:33 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:33.305570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 782/1588] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:33 2022] POST /identity/v3/users => generated 278 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:33.305682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c558c-c4c3-4342-92b4-9646e025ef95 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.305768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e9802-b8f9-40dc-98c2-440fda02ac92 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.305768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e9802-b8f9-40dc-98c2-440fda02ac92 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.305768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e9802-b8f9-40dc-98c2-440fda02ac92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.319389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c558c-c4c3-4342-92b4-9646e025ef95 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.325215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461c161-7eb3-4727-b9f0-cc1e4d1fd62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.325948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461c161-7eb3-4727-b9f0-cc1e4d1fd62b admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.326261 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461c161-7eb3-4727-b9f0-cc1e4d1fd62b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.326571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461c161-7eb3-4727-b9f0-cc1e4d1fd62b admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.328384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461c161-7eb3-4727-b9f0-cc1e4d1fd62b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.330515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461c161-7eb3-4727-b9f0-cc1e4d1fd62b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.336870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-9d7e9802-b8f9-40dc-98c2-440fda02ac92 None None] MFA Rules not processed for user `33c8564825094c4ca91bd2511fd3e425`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:33.338318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9d7e9802-b8f9-40dc-98c2-440fda02ac92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:33.419123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 783/1589] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:33 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:33.419888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e9802-b8f9-40dc-98c2-440fda02ac92 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.426921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:33.431272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e9802-b8f9-40dc-98c2-440fda02ac92 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.437847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aaa6d-f4b6-4475-9fd5-228b1b2b7023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.438719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aaa6d-f4b6-4475-9fd5-228b1b2b7023 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.439547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aaa6d-f4b6-4475-9fd5-228b1b2b7023 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.439547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aaa6d-f4b6-4475-9fd5-228b1b2b7023 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.443695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321aaa6d-f4b6-4475-9fd5-228b1b2b7023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.443811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 807/1590] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:33 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:33.443911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321aaa6d-f4b6-4475-9fd5-228b1b2b7023 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.444006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461c161-7eb3-4727-b9f0-cc1e4d1fd62b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.453704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461c161-7eb3-4727-b9f0-cc1e4d1fd62b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.457975 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 784/1591] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:33.457975 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aaa6d-f4b6-4475-9fd5-228b1b2b7023 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.462510 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c5ecf-e628-4c4e-a68e-94f5ae8756d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.463297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c5ecf-e628-4c4e-a68e-94f5ae8756d3 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.463582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c5ecf-e628-4c4e-a68e-94f5ae8756d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.463857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c5ecf-e628-4c4e-a68e-94f5ae8756d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.464591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c5ecf-e628-4c4e-a68e-94f5ae8756d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.466230 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c5ecf-e628-4c4e-a68e-94f5ae8756d3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.467930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aaa6d-f4b6-4475-9fd5-228b1b2b7023 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.474085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fd8f7-f6da-484e-8c9d-25b38c97f60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.474868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fd8f7-f6da-484e-8c9d-25b38c97f60c admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.475186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fd8f7-f6da-484e-8c9d-25b38c97f60c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.475471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fd8f7-f6da-484e-8c9d-25b38c97f60c admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.476326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fd8f7-f6da-484e-8c9d-25b38c97f60c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.477928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fd8f7-f6da-484e-8c9d-25b38c97f60c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.481237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 808/1592] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:33.482136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c5ecf-e628-4c4e-a68e-94f5ae8756d3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.491518 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 785/1593] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:33.491907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fd8f7-f6da-484e-8c9d-25b38c97f60c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.492467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c5ecf-e628-4c4e-a68e-94f5ae8756d3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.502260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad17728-4537-472b-a1a8-3a8e9f75860c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.503119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad17728-4537-472b-a1a8-3a8e9f75860c admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.503463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad17728-4537-472b-a1a8-3a8e9f75860c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.503813 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad17728-4537-472b-a1a8-3a8e9f75860c admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.504678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad17728-4537-472b-a1a8-3a8e9f75860c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.504795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fd8f7-f6da-484e-8c9d-25b38c97f60c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.506546 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad17728-4537-472b-a1a8-3a8e9f75860c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.510847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bac2b1-1ae1-46e8-b487-e5f0d372736e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.511397 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bac2b1-1ae1-46e8-b487-e5f0d372736e admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.511745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bac2b1-1ae1-46e8-b487-e5f0d372736e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.512087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bac2b1-1ae1-46e8-b487-e5f0d372736e admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.513262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bac2b1-1ae1-46e8-b487-e5f0d372736e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.514950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bac2b1-1ae1-46e8-b487-e5f0d372736e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.519239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 809/1594] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:33.520192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad17728-4537-472b-a1a8-3a8e9f75860c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.531309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad17728-4537-472b-a1a8-3a8e9f75860c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.533547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 786/1595] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:33.534467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bac2b1-1ae1-46e8-b487-e5f0d372736e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.539947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a2a14-a414-419b-96cf-ca6629869928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.541145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a2a14-a414-419b-96cf-ca6629869928 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.541145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a2a14-a414-419b-96cf-ca6629869928 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.541602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a2a14-a414-419b-96cf-ca6629869928 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.543391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a2a14-a414-419b-96cf-ca6629869928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.544458 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a2a14-a414-419b-96cf-ca6629869928 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.553891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bac2b1-1ae1-46e8-b487-e5f0d372736e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.558157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad55f0c-eb95-49bf-b7a4-9dfec87896e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.559049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad55f0c-eb95-49bf-b7a4-9dfec87896e1 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.560586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad55f0c-eb95-49bf-b7a4-9dfec87896e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.560586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad55f0c-eb95-49bf-b7a4-9dfec87896e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.561439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad55f0c-eb95-49bf-b7a4-9dfec87896e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.563815 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad55f0c-eb95-49bf-b7a4-9dfec87896e1 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.576385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 787/1596] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:33.578612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad55f0c-eb95-49bf-b7a4-9dfec87896e1 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.590015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad55f0c-eb95-49bf-b7a4-9dfec87896e1 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.596527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076b36e-c5db-45ec-a3fb-1597e7adc616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.597331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076b36e-c5db-45ec-a3fb-1597e7adc616 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.597704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076b36e-c5db-45ec-a3fb-1597e7adc616 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.598048 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076b36e-c5db-45ec-a3fb-1597e7adc616 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.598882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076b36e-c5db-45ec-a3fb-1597e7adc616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.601502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076b36e-c5db-45ec-a3fb-1597e7adc616 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.611819 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 788/1597] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:33.612747 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076b36e-c5db-45ec-a3fb-1597e7adc616 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.622966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076b36e-c5db-45ec-a3fb-1597e7adc616 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.628889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a60349-2934-4b12-9e11-9914bd2329f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.629972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a60349-2934-4b12-9e11-9914bd2329f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.630106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a60349-2934-4b12-9e11-9914bd2329f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.630322 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a60349-2934-4b12-9e11-9914bd2329f5 admin admin] PATH_INFO: `/v3/projects/850db3538ad34024a004f5d3cea7d64e/users/bbe7dec7280541cf98c3ab6f56b06c7e/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.651110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 810/1598] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:33 2022] POST /identity/v3/projects => generated 406 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:33.651340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a60349-2934-4b12-9e11-9914bd2329f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=850db3538ad34024a004f5d3cea7d64e, user_id=bbe7dec7280541cf98c3ab6f56b06c7e, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.651468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a2a14-a414-419b-96cf-ca6629869928 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.651567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a60349-2934-4b12-9e11-9914bd2329f5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.718943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a2a14-a414-419b-96cf-ca6629869928 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.723599 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69a5f7-c230-4ba2-99b7-d63482523920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.724486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69a5f7-c230-4ba2-99b7-d63482523920 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.724765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69a5f7-c230-4ba2-99b7-d63482523920 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.725038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69a5f7-c230-4ba2-99b7-d63482523920 admin admin] PATH_INFO: `/v3/projects/6251e828bb784b898ecf49686477f863/users/146896e42498467b9b42cb49d73db2e8/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.742377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69a5f7-c230-4ba2-99b7-d63482523920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6251e828bb784b898ecf49686477f863, user_id=146896e42498467b9b42cb49d73db2e8, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.745488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69a5f7-c230-4ba2-99b7-d63482523920 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.764057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 789/1599] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:33 2022] PUT /identity/v3/projects/850db3538ad34024a004f5d3cea7d64e/users/bbe7dec7280541cf98c3ab6f56b06c7e/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:33.769860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a60349-2934-4b12-9e11-9914bd2329f5 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.828848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 811/1600] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:33 2022] PUT /identity/v3/projects/6251e828bb784b898ecf49686477f863/users/146896e42498467b9b42cb49d73db2e8/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:33.834894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69a5f7-c230-4ba2-99b7-d63482523920 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.837974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a60349-2934-4b12-9e11-9914bd2329f5 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.875865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:33.913846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb447f8d-24ad-44b9-ab98-6b7f981d7979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.914577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb447f8d-24ad-44b9-ab98-6b7f981d7979 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.914882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb447f8d-24ad-44b9-ab98-6b7f981d7979 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.915163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb447f8d-24ad-44b9-ab98-6b7f981d7979 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.915916 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb447f8d-24ad-44b9-ab98-6b7f981d7979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.920963 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb447f8d-24ad-44b9-ab98-6b7f981d7979 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.927763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69a5f7-c230-4ba2-99b7-d63482523920 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.929836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 790/1601] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/roles => generated 1427 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:33.930561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb447f8d-24ad-44b9-ab98-6b7f981d7979 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.934094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fea3a-7057-4e03-928c-052e605ac2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.934560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fea3a-7057-4e03-928c-052e605ac2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.934851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fea3a-7057-4e03-928c-052e605ac2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.935135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fea3a-7057-4e03-928c-052e605ac2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.936607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160fea3a-7057-4e03-928c-052e605ac2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.938913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160fea3a-7057-4e03-928c-052e605ac2f3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.941284 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb447f8d-24ad-44b9-ab98-6b7f981d7979 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.945945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d32a7-4af2-4ac9-aa58-3f3cb3e18bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.946622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d32a7-4af2-4ac9-aa58-3f3cb3e18bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.946931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d32a7-4af2-4ac9-aa58-3f3cb3e18bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.947208 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d32a7-4af2-4ac9-aa58-3f3cb3e18bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.947936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d32a7-4af2-4ac9-aa58-3f3cb3e18bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.949530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d32a7-4af2-4ac9-aa58-3f3cb3e18bb0 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.950370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 812/1602] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/roles => generated 1427 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:33.955076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fea3a-7057-4e03-928c-052e605ac2f3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.963417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 791/1603] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:33 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:33.964466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d32a7-4af2-4ac9-aa58-3f3cb3e18bb0 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:33.965245 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fea3a-7057-4e03-928c-052e605ac2f3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.970045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ac8a2-ee2c-4cb6-8284-651096dcb319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.970892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ac8a2-ee2c-4cb6-8284-651096dcb319 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.971181 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ac8a2-ee2c-4cb6-8284-651096dcb319 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.971464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ac8a2-ee2c-4cb6-8284-651096dcb319 admin admin] PATH_INFO: `/v3/projects/69b110a601c541328917618364dcad5b/users/4ffcb99923b346938eb72c0b2786e815/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.976893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d32a7-4af2-4ac9-aa58-3f3cb3e18bb0 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:33.981609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b0848-d056-4953-9b89-27eeea7814f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:33.982276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b0848-d056-4953-9b89-27eeea7814f4 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:33.982561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b0848-d056-4953-9b89-27eeea7814f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:33.982874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b0848-d056-4953-9b89-27eeea7814f4 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:33.983956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b0848-d056-4953-9b89-27eeea7814f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.985857 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b0848-d056-4953-9b89-27eeea7814f4 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:33.987122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ac8a2-ee2c-4cb6-8284-651096dcb319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b110a601c541328917618364dcad5b, user_id=4ffcb99923b346938eb72c0b2786e815, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:33.989896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ac8a2-ee2c-4cb6-8284-651096dcb319 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:34.088705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 813/1604] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:33 2022] PUT /identity/v3/projects/69b110a601c541328917618364dcad5b/users/4ffcb99923b346938eb72c0b2786e815/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:34.088705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1644fdf-1419-4b86-a923-e57a012509ae None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.088705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1644fdf-1419-4b86-a923-e57a012509ae None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.088705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1644fdf-1419-4b86-a923-e57a012509ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.092327 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:34.107890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 792/1605] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:33 2022] POST /identity/v3/users => generated 310 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:34.107890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89312837-2777-443e-8419-069ac107a72e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.107890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89312837-2777-443e-8419-069ac107a72e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.107890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89312837-2777-443e-8419-069ac107a72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.112181 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b1644fdf-1419-4b86-a923-e57a012509ae None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:34.113616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b1644fdf-1419-4b86-a923-e57a012509ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:34.141303 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-89312837-2777-443e-8419-069ac107a72e None None] MFA Rules not processed for user `146896e42498467b9b42cb49d73db2e8`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:34.142073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-89312837-2777-443e-8419-069ac107a72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:34.189326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 814/1606] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:34.190221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1644fdf-1419-4b86-a923-e57a012509ae None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:34.203671 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1644fdf-1419-4b86-a923-e57a012509ae None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:34.209695 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9bd9f-c776-45bf-9c82-0ae8238254d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:34.210209 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9bd9f-c776-45bf-9c82-0ae8238254d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.210494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9bd9f-c776-45bf-9c82-0ae8238254d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.210770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9bd9f-c776-45bf-9c82-0ae8238254d0 admin admin] PATH_INFO: `/v3/projects/68bd0110e4fd4e6dbef9f65397908292/users/fe1c86c411834f1c8d2775a5b2c5f1d1/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.229086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9bd9f-c776-45bf-9c82-0ae8238254d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68bd0110e4fd4e6dbef9f65397908292, user_id=fe1c86c411834f1c8d2775a5b2c5f1d1, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:34.230469 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 793/1607] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:34.231164 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89312837-2777-443e-8419-069ac107a72e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:34.232570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9bd9f-c776-45bf-9c82-0ae8238254d0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:34.245246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89312837-2777-443e-8419-069ac107a72e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:34.250421 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad18b18-996d-4ab1-8d66-00a8704a9f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:34.255996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad18b18-996d-4ab1-8d66-00a8704a9f9e admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.255996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad18b18-996d-4ab1-8d66-00a8704a9f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.255996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad18b18-996d-4ab1-8d66-00a8704a9f9e admin admin] PATH_INFO: `/v3/projects/f44ac7096ce24241ac9984676d801b52/users/add6b90e58e943579c0f152f25a77e2a/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.269589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad18b18-996d-4ab1-8d66-00a8704a9f9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44ac7096ce24241ac9984676d801b52, user_id=add6b90e58e943579c0f152f25a77e2a, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:34.272375 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad18b18-996d-4ab1-8d66-00a8704a9f9e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:34.353678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 815/1608] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:34 2022] PUT /identity/v3/projects/68bd0110e4fd4e6dbef9f65397908292/users/fe1c86c411834f1c8d2775a5b2c5f1d1/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:34.354537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9bd9f-c776-45bf-9c82-0ae8238254d0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:34.425893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 794/1609] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:34 2022] PUT /identity/v3/projects/f44ac7096ce24241ac9984676d801b52/users/add6b90e58e943579c0f152f25a77e2a/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:34.427702 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67865c-3199-486c-9320-5a1d02bb0c40 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.428511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67865c-3199-486c-9320-5a1d02bb0c40 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.428511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67865c-3199-486c-9320-5a1d02bb0c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.442447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9bd9f-c776-45bf-9c82-0ae8238254d0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:34.446779 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1b67865c-3199-486c-9320-5a1d02bb0c40 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:34.448078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1b67865c-3199-486c-9320-5a1d02bb0c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:34.458228 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:34.507158 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf44ff7-cd76-4bd9-843e-2ec9d32f6035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:34.507965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf44ff7-cd76-4bd9-843e-2ec9d32f6035 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.508243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf44ff7-cd76-4bd9-843e-2ec9d32f6035 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.508544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf44ff7-cd76-4bd9-843e-2ec9d32f6035 admin admin] PATH_INFO: `/v3/projects/d338e1d056d94062a703aa03ecb7edbb/users/84b4d6be32e94ccd80f97aa8c93efbcc/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.513731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 795/1610] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:34.516417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67865c-3199-486c-9320-5a1d02bb0c40 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:34.527040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf44ff7-cd76-4bd9-843e-2ec9d32f6035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d338e1d056d94062a703aa03ecb7edbb, user_id=84b4d6be32e94ccd80f97aa8c93efbcc, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:34.530416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf44ff7-cd76-4bd9-843e-2ec9d32f6035 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:34.530574 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67865c-3199-486c-9320-5a1d02bb0c40 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:34.536954 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558a612-7b2d-42fd-96f9-13eedac8dc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:34.537631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558a612-7b2d-42fd-96f9-13eedac8dc81 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.537912 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558a612-7b2d-42fd-96f9-13eedac8dc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.538186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558a612-7b2d-42fd-96f9-13eedac8dc81 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.538988 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558a612-7b2d-42fd-96f9-13eedac8dc81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:34.540644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558a612-7b2d-42fd-96f9-13eedac8dc81 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:34.552179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 796/1611] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:34 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:34.553152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558a612-7b2d-42fd-96f9-13eedac8dc81 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:34.621914 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558a612-7b2d-42fd-96f9-13eedac8dc81 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:34.626439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0e288-a8ea-4037-9b87-a8f5b518845e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:34.627091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0e288-a8ea-4037-9b87-a8f5b518845e service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.627377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0e288-a8ea-4037-9b87-a8f5b518845e service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.627651 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0e288-a8ea-4037-9b87-a8f5b518845e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.640681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0e288-a8ea-4037-9b87-a8f5b518845e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:34.642601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0e288-a8ea-4037-9b87-a8f5b518845e service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:34.648389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:34.648707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 816/1612] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:34 2022] PUT /identity/v3/projects/d338e1d056d94062a703aa03ecb7edbb/users/84b4d6be32e94ccd80f97aa8c93efbcc/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:34.650130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077c220-cef5-4fc7-ab71-18f675c150fd None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.650412 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077c220-cef5-4fc7-ab71-18f675c150fd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.650681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077c220-cef5-4fc7-ab71-18f675c150fd None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.653309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 817/1613] 10.222.0.121 () {54 vars in 966 bytes} [Thu May 26 17:59:34 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:59:34.654544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9f854-e424-4d32-9432-cc56ea41ed3c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.655064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9f854-e424-4d32-9432-cc56ea41ed3c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.655878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9f854-e424-4d32-9432-cc56ea41ed3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.674799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 797/1614] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:34.677591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23baa7f-0d93-4d88-9d6f-455694aac757 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.678195 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23baa7f-0d93-4d88-9d6f-455694aac757 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.678583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23baa7f-0d93-4d88-9d6f-455694aac757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.692294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-42d9f854-e424-4d32-9432-cc56ea41ed3c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:34.693617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-42d9f854-e424-4d32-9432-cc56ea41ed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:34.714595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b23baa7f-0d93-4d88-9d6f-455694aac757 None None] MFA Rules not processed for user `add6b90e58e943579c0f152f25a77e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:34.716518 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b23baa7f-0d93-4d88-9d6f-455694aac757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:34.795234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 818/1615] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:34.796382 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9f854-e424-4d32-9432-cc56ea41ed3c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:34.798039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-42d9f854-e424-4d32-9432-cc56ea41ed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:34.821731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 798/1616] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:34 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:34.822736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23baa7f-0d93-4d88-9d6f-455694aac757 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:34.834584 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23baa7f-0d93-4d88-9d6f-455694aac757 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:34.839814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866436b3-3bcc-4022-93ca-462ae0c67e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:34.840568 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866436b3-3bcc-4022-93ca-462ae0c67e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.840842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866436b3-3bcc-4022-93ca-462ae0c67e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.841105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866436b3-3bcc-4022-93ca-462ae0c67e74 admin admin] PATH_INFO: `/v3/projects/c1e2a3642be04d908265413df099b8f1/users/14afd27854224e758b4336515f628161/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.871767 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866436b3-3bcc-4022-93ca-462ae0c67e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e2a3642be04d908265413df099b8f1, user_id=14afd27854224e758b4336515f628161, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:34.872024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9f854-e424-4d32-9432-cc56ea41ed3c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:34.875385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866436b3-3bcc-4022-93ca-462ae0c67e74 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:34.876655 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c99508-4862-45f0-b184-401fb2d026c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:34.877443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c99508-4862-45f0-b184-401fb2d026c4 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.877806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c99508-4862-45f0-b184-401fb2d026c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.878151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c99508-4862-45f0-b184-401fb2d026c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.888676 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c99508-4862-45f0-b184-401fb2d026c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:34.892684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c99508-4862-45f0-b184-401fb2d026c4 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:34.901406 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 819/1617] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:34.902936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e72bc-c242-459e-a5a4-086bd623fac2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.903400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e72bc-c242-459e-a5a4-086bd623fac2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.903679 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e72bc-c242-459e-a5a4-086bd623fac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.927256 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b11e72bc-c242-459e-a5a4-086bd623fac2 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:34.928349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b11e72bc-c242-459e-a5a4-086bd623fac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:34.941221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 820/1618] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:34.941221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97c340-6a93-46de-b75c-97d686b64055 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:34.941221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97c340-6a93-46de-b75c-97d686b64055 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:34.941221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97c340-6a93-46de-b75c-97d686b64055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:34.962039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-de97c340-6a93-46de-b75c-97d686b64055 None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:34.963523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-de97c340-6a93-46de-b75c-97d686b64055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:34.982560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 799/1619] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:34 2022] PUT /identity/v3/projects/c1e2a3642be04d908265413df099b8f1/users/14afd27854224e758b4336515f628161/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:34.984227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866436b3-3bcc-4022-93ca-462ae0c67e74 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:35.009020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:35.074682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866436b3-3bcc-4022-93ca-462ae0c67e74 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:35.078842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 821/1620] 10.222.0.121 () {58 vars in 1063 bytes} [Thu May 26 17:59:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:35.079294 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a530403-22b5-4440-a36b-f82c4abb9010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:35.080126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97c340-6a93-46de-b75c-97d686b64055 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:35.080257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a530403-22b5-4440-a36b-f82c4abb9010 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:35.080632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a530403-22b5-4440-a36b-f82c4abb9010 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:35.081026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a530403-22b5-4440-a36b-f82c4abb9010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:35.107807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97c340-6a93-46de-b75c-97d686b64055 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:35.196346 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a530403-22b5-4440-a36b-f82c4abb9010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:35.198218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a530403-22b5-4440-a36b-f82c4abb9010 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:35.212531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 800/1621] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:35.213706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a530403-22b5-4440-a36b-f82c4abb9010 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:35.226531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a530403-22b5-4440-a36b-f82c4abb9010 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:35.230184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940a2b3-79a8-4eca-8834-2c9a94d20f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:35.231529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940a2b3-79a8-4eca-8834-2c9a94d20f4f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:35.231818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940a2b3-79a8-4eca-8834-2c9a94d20f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:35.232096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940a2b3-79a8-4eca-8834-2c9a94d20f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:35.232842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e08e1f-0ee5-4d08-b478-f06418467bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:35.234279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e08e1f-0ee5-4d08-b478-f06418467bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:35.234892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e08e1f-0ee5-4d08-b478-f06418467bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:35.235651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940a2b3-79a8-4eca-8834-2c9a94d20f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:35.235792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e08e1f-0ee5-4d08-b478-f06418467bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:35.236320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e08e1f-0ee5-4d08-b478-f06418467bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:35.237426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940a2b3-79a8-4eca-8834-2c9a94d20f4f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:35.238060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e08e1f-0ee5-4d08-b478-f06418467bb3 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:35.254869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 801/1622] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:35 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:35.255633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e08e1f-0ee5-4d08-b478-f06418467bb3 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:35.266675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:35.275758 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e08e1f-0ee5-4d08-b478-f06418467bb3 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:35.279723 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2073cc-5f7e-4f6c-9712-b9d68a108fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:35.281727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2073cc-5f7e-4f6c-9712-b9d68a108fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:35.282156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2073cc-5f7e-4f6c-9712-b9d68a108fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:35.282585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2073cc-5f7e-4f6c-9712-b9d68a108fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:35.293074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2073cc-5f7e-4f6c-9712-b9d68a108fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:35.294616 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2073cc-5f7e-4f6c-9712-b9d68a108fe0 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:35.318281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 802/1623] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:35.318281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2073cc-5f7e-4f6c-9712-b9d68a108fe0 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:35.398755 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 822/1624] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:35 2022] POST /identity/v3/projects => generated 358 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:35.399856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940a2b3-79a8-4eca-8834-2c9a94d20f4f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:35.455299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2073cc-5f7e-4f6c-9712-b9d68a108fe0 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:35.518532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940a2b3-79a8-4eca-8834-2c9a94d20f4f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:35.526653 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ecbc9-633d-43d5-bff1-b5d84a3acaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:35.527819 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ecbc9-633d-43d5-bff1-b5d84a3acaa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:35.528587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ecbc9-633d-43d5-bff1-b5d84a3acaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:35.528881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ecbc9-633d-43d5-bff1-b5d84a3acaa5 admin admin] PATH_INFO: `/v3/projects/c1e2a3642be04d908265413df099b8f1/users/14afd27854224e758b4336515f628161/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:35.532537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ecbc9-633d-43d5-bff1-b5d84a3acaa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e2a3642be04d908265413df099b8f1, user_id=14afd27854224e758b4336515f628161, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:35.536639 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ecbc9-633d-43d5-bff1-b5d84a3acaa5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:35.543425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5d424-0cf6-419b-a4ad-d45b789b49ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:35.544441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5d424-0cf6-419b-a4ad-d45b789b49ad service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:35.544957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5d424-0cf6-419b-a4ad-d45b789b49ad service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:35.545612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5d424-0cf6-419b-a4ad-d45b789b49ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:35.560013 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 823/1625] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:35 2022] PUT /identity/v3/projects/c1e2a3642be04d908265413df099b8f1/users/14afd27854224e758b4336515f628161/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:35.562516 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ecbc9-633d-43d5-bff1-b5d84a3acaa5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:35.634131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5d424-0cf6-419b-a4ad-d45b789b49ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:35.641424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5d424-0cf6-419b-a4ad-d45b789b49ad service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:35.644293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:35.659454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 803/1626] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:35.661194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5a324-33c1-4b66-80e8-ef1280fb564a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:35.662568 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5a324-33c1-4b66-80e8-ef1280fb564a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:35.662997 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5a324-33c1-4b66-80e8-ef1280fb564a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:35.665749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ecbc9-633d-43d5-bff1-b5d84a3acaa5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:35.671935 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cbbf7-99c9-41ca-9b8b-d89c6533d4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:35.672744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cbbf7-99c9-41ca-9b8b-d89c6533d4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:35.673360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cbbf7-99c9-41ca-9b8b-d89c6533d4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:35.673706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cbbf7-99c9-41ca-9b8b-d89c6533d4b6 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:35.675022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cbbf7-99c9-41ca-9b8b-d89c6533d4b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:35.677346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cbbf7-99c9-41ca-9b8b-d89c6533d4b6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:35.728782 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d5e5a324-33c1-4b66-80e8-ef1280fb564a None None] MFA Rules not processed for user `14afd27854224e758b4336515f628161`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:35.730156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d5e5a324-33c1-4b66-80e8-ef1280fb564a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:35.765744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:35.817960 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 824/1627] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:35 2022] POST /identity/v3/users => generated 286 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:35.822589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cbbf7-99c9-41ca-9b8b-d89c6533d4b6 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:35.840465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cbbf7-99c9-41ca-9b8b-d89c6533d4b6 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:35.850093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af33a97-2f67-475e-96a7-5674cad47014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:35.850963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af33a97-2f67-475e-96a7-5674cad47014 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:35.851978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af33a97-2f67-475e-96a7-5674cad47014 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:35.852331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af33a97-2f67-475e-96a7-5674cad47014 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:35.854171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af33a97-2f67-475e-96a7-5674cad47014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:35.861917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af33a97-2f67-475e-96a7-5674cad47014 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:35.873504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 804/1628] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:35 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:35.888409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5a324-33c1-4b66-80e8-ef1280fb564a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:35.891253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 825/1629] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:35 2022] GET /identity/v3/roles => generated 1427 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:59:35.895805 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af33a97-2f67-475e-96a7-5674cad47014 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:35.921329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af33a97-2f67-475e-96a7-5674cad47014 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:35.929262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098356-ddab-4558-9bf9-9a0b4917a01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:35.937411 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098356-ddab-4558-9bf9-9a0b4917a01b admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:35.937411 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098356-ddab-4558-9bf9-9a0b4917a01b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:35.937411 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098356-ddab-4558-9bf9-9a0b4917a01b admin admin] PATH_INFO: `/v3/projects/81ca2bbff5914bed90929efd7e41faf9/users/61a17846d01f4107a6aaa60c876bd26f/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:35.967015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed098356-ddab-4558-9bf9-9a0b4917a01b admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ca2bbff5914bed90929efd7e41faf9, user_id=61a17846d01f4107a6aaa60c876bd26f, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:35.971740 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed098356-ddab-4558-9bf9-9a0b4917a01b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:35.989361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5a324-33c1-4b66-80e8-ef1280fb564a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:35.994620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443a2cf-5801-4a66-ba49-48592f1b8d12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:35.995351 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443a2cf-5801-4a66-ba49-48592f1b8d12 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:35.995629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443a2cf-5801-4a66-ba49-48592f1b8d12 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:35.996471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443a2cf-5801-4a66-ba49-48592f1b8d12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:36.008851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443a2cf-5801-4a66-ba49-48592f1b8d12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:36.011191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443a2cf-5801-4a66-ba49-48592f1b8d12 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:36.036620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 805/1630] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:35 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:36.057043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd3099-8471-437b-8455-e5df191c60b4 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:36.057390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd3099-8471-437b-8455-e5df191c60b4 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:36.057678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd3099-8471-437b-8455-e5df191c60b4 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:36.059310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 806/1631] 10.222.0.121 () {54 vars in 917 bytes} [Thu May 26 17:59:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:59:36.067090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059be1-4f2e-4b86-bd1c-879ff8066bf1 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:36.067395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059be1-4f2e-4b86-bd1c-879ff8066bf1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:36.067669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059be1-4f2e-4b86-bd1c-879ff8066bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:36.099926 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d5059be1-4f2e-4b86-bd1c-879ff8066bf1 None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:36.101146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d5059be1-4f2e-4b86-bd1c-879ff8066bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:36.172254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 826/1632] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:35 2022] PUT /identity/v3/projects/81ca2bbff5914bed90929efd7e41faf9/users/61a17846d01f4107a6aaa60c876bd26f/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:59:36.179072 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098356-ddab-4558-9bf9-9a0b4917a01b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:36.414552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 807/1633] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 17:59:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:36.431882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059be1-4f2e-4b86-bd1c-879ff8066bf1 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:36.452945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059be1-4f2e-4b86-bd1c-879ff8066bf1 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:36.453304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098356-ddab-4558-9bf9-9a0b4917a01b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:36.480857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027ada8-a64c-458f-b1fa-de36df316a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:36.481574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027ada8-a64c-458f-b1fa-de36df316a64 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:36.481905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027ada8-a64c-458f-b1fa-de36df316a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:36.482220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027ada8-a64c-458f-b1fa-de36df316a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:36.483060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027ada8-a64c-458f-b1fa-de36df316a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:36.485707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027ada8-a64c-458f-b1fa-de36df316a64 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:36.504253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 827/1634] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:36 2022] GET /identity/v3/roles => generated 1427 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:36.515785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027ada8-a64c-458f-b1fa-de36df316a64 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:36.547326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027ada8-a64c-458f-b1fa-de36df316a64 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:36.559577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2807b-ae42-441f-9701-b325ca276c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:36.560559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2807b-ae42-441f-9701-b325ca276c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:36.561316 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2807b-ae42-441f-9701-b325ca276c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:36.561619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2807b-ae42-441f-9701-b325ca276c22 admin admin] PATH_INFO: `/v3/projects/81ca2bbff5914bed90929efd7e41faf9/users/61a17846d01f4107a6aaa60c876bd26f/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:36.564533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2807b-ae42-441f-9701-b325ca276c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ca2bbff5914bed90929efd7e41faf9, user_id=61a17846d01f4107a6aaa60c876bd26f, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:36.567465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2807b-ae42-441f-9701-b325ca276c22 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:36.582907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 828/1635] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:36 2022] PUT /identity/v3/projects/81ca2bbff5914bed90929efd7e41faf9/users/61a17846d01f4107a6aaa60c876bd26f/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:36.584441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2807b-ae42-441f-9701-b325ca276c22 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:36.594977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:36.615686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c10b6-4312-4788-af1f-3e2cf00cc8aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:36.616530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c10b6-4312-4788-af1f-3e2cf00cc8aa service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:36.619367 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c10b6-4312-4788-af1f-3e2cf00cc8aa service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:36.624976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c10b6-4312-4788-af1f-3e2cf00cc8aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:36.721770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2807b-ae42-441f-9701-b325ca276c22 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:36.728136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:36.729393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:36.729885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:36.730326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] PATH_INFO: `/v3/users/33c8564825094c4ca91bd2511fd3e425` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:36.732524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c8564825094c4ca91bd2511fd3e425)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:36.735590 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:36.796329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c10b6-4312-4788-af1f-3e2cf00cc8aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:36.800235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c10b6-4312-4788-af1f-3e2cf00cc8aa service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:36.828403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 808/1636] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:36.830593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4877a-a9a2-46e7-afa4-6829c78e70d2 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:36.830935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4877a-a9a2-46e7-afa4-6829c78e70d2 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:36.832780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4877a-a9a2-46e7-afa4-6829c78e70d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:36.861528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:36.899845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-30a4877a-a9a2-46e7-afa4-6829c78e70d2 None None] MFA Rules not processed for user `61a17846d01f4107a6aaa60c876bd26f`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:36.902368 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-30a4877a-a9a2-46e7-afa4-6829c78e70d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:36.973674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c8564825094c4ca91bd2511fd3e425'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:36.996680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c8564825094c4ca91bd2511fd3e425'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:36.997682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c8564825094c4ca91bd2511fd3e425'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:37.019173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c8564825094c4ca91bd2511fd3e425'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:37.054419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 809/1637] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:36 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:37.056768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c313d2-f860-490d-9dd6-bedb29ceb52e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:37.057209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c313d2-f860-490d-9dd6-bedb29ceb52e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:37.057635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c313d2-f860-490d-9dd6-bedb29ceb52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:37.335426 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a1c313d2-f860-490d-9dd6-bedb29ceb52e None None] MFA Rules not processed for user `bbe7dec7280541cf98c3ab6f56b06c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:37.335426 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a1c313d2-f860-490d-9dd6-bedb29ceb52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:37.339532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c8564825094c4ca91bd2511fd3e425'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:37.339532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 829/1638] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 17:59:36 2022] DELETE /identity/v3/users/33c8564825094c4ca91bd2511fd3e425 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:37.339532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:37.339532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:37.340239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:37.340239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 810/1639] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:37 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:37.340239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c34153-89a5-4170-8397-896c0e48d51f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:37.340239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c34153-89a5-4170-8397-896c0e48d51f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:37.340239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c34153-89a5-4170-8397-896c0e48d51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:37.346384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66652e3e-0e83-424b-85e8-8b395acb710a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:37.357414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767b1a0-fc8e-4bb4-8ad2-f5a92e67a6f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:37.358313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767b1a0-fc8e-4bb4-8ad2-f5a92e67a6f3 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:37.359611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767b1a0-fc8e-4bb4-8ad2-f5a92e67a6f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:37.359945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767b1a0-fc8e-4bb4-8ad2-f5a92e67a6f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:37.367235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5767b1a0-fc8e-4bb4-8ad2-f5a92e67a6f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:37.397692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-73c34153-89a5-4170-8397-896c0e48d51f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:37.399190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-73c34153-89a5-4170-8397-896c0e48d51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:37.520591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 811/1640] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:37.522982 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c34153-89a5-4170-8397-896c0e48d51f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:37.531205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-73c34153-89a5-4170-8397-896c0e48d51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:37.545998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767b1a0-fc8e-4bb4-8ad2-f5a92e67a6f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:37.545998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767b1a0-fc8e-4bb4-8ad2-f5a92e67a6f3 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:37.553535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c34153-89a5-4170-8397-896c0e48d51f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:37.555898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 830/1641] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:37.557240 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767b1a0-fc8e-4bb4-8ad2-f5a92e67a6f3 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:37.558797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5767b1a0-fc8e-4bb4-8ad2-f5a92e67a6f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:37.559088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710d5ff-61fa-4432-b551-4f7b60ac72dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:37.560041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710d5ff-61fa-4432-b551-4f7b60ac72dc admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:37.560329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710d5ff-61fa-4432-b551-4f7b60ac72dc admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:37.560718 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710d5ff-61fa-4432-b551-4f7b60ac72dc admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:37.561948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710d5ff-61fa-4432-b551-4f7b60ac72dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:37.564345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710d5ff-61fa-4432-b551-4f7b60ac72dc admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:37.724330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 812/1642] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:37 2022] POST /identity/v3/projects => generated 402 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:37.725508 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710d5ff-61fa-4432-b551-4f7b60ac72dc admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:37.793983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767b1a0-fc8e-4bb4-8ad2-f5a92e67a6f3 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:37.919825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a3a18-062a-4bb1-b5b7-2679811bcdd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:37.921024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a3a18-062a-4bb1-b5b7-2679811bcdd6 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:37.921369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a3a18-062a-4bb1-b5b7-2679811bcdd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:37.922303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a3a18-062a-4bb1-b5b7-2679811bcdd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:37.929174 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710d5ff-61fa-4432-b551-4f7b60ac72dc admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:37.936676 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a3a18-062a-4bb1-b5b7-2679811bcdd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:37.938192 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2f291-40dc-4c51-a959-f2fdb7a17b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:37.938319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a3a18-062a-4bb1-b5b7-2679811bcdd6 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:37.939541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2f291-40dc-4c51-a959-f2fdb7a17b13 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:37.940028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2f291-40dc-4c51-a959-f2fdb7a17b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:37.940433 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2f291-40dc-4c51-a959-f2fdb7a17b13 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:37.941308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2f291-40dc-4c51-a959-f2fdb7a17b13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:37.943667 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2f291-40dc-4c51-a959-f2fdb7a17b13 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:37.950188 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 831/1643] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:38.041714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a3a18-062a-4bb1-b5b7-2679811bcdd6 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:38.044691 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 813/1644] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:37 2022] POST /identity/v3/users => generated 308 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:38.051274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2f291-40dc-4c51-a959-f2fdb7a17b13 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:38.069706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2f291-40dc-4c51-a959-f2fdb7a17b13 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:38.092291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2b8f3-99ac-4132-bbcd-cf6b06b15616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:38.093960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2b8f3-99ac-4132-bbcd-cf6b06b15616 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:38.093960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2b8f3-99ac-4132-bbcd-cf6b06b15616 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:38.094199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2b8f3-99ac-4132-bbcd-cf6b06b15616 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:38.094966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a2b8f3-99ac-4132-bbcd-cf6b06b15616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:38.099814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a2b8f3-99ac-4132-bbcd-cf6b06b15616 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:38.128725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 814/1645] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:38 2022] GET /identity/v3/roles => generated 1427 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:59:38.134488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2b8f3-99ac-4132-bbcd-cf6b06b15616 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:38.165266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2b8f3-99ac-4132-bbcd-cf6b06b15616 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:38.174286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8612bbf-a3e3-4e7d-8317-e59ff0b57522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:38.175312 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8612bbf-a3e3-4e7d-8317-e59ff0b57522 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:38.175672 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8612bbf-a3e3-4e7d-8317-e59ff0b57522 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:38.176030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8612bbf-a3e3-4e7d-8317-e59ff0b57522 admin admin] PATH_INFO: `/v3/projects/23b3bfaaf2974d51a6737b0e5efce913/users/a8f2c9e2cb994438a6db6860a525b264/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:38.202111 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8612bbf-a3e3-4e7d-8317-e59ff0b57522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b3bfaaf2974d51a6737b0e5efce913, user_id=a8f2c9e2cb994438a6db6860a525b264, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:38.205807 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8612bbf-a3e3-4e7d-8317-e59ff0b57522 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:38.247281 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a3a18-062a-4bb1-b5b7-2679811bcdd6 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:38.262367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9c9da-5343-4bb4-a5b1-dd14d5b6d90f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:38.263663 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9c9da-5343-4bb4-a5b1-dd14d5b6d90f service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:38.266279 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9c9da-5343-4bb4-a5b1-dd14d5b6d90f service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:38.266733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9c9da-5343-4bb4-a5b1-dd14d5b6d90f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:38.275128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5da9c9da-5343-4bb4-a5b1-dd14d5b6d90f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:38.321729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 815/1646] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:38 2022] PUT /identity/v3/projects/23b3bfaaf2974d51a6737b0e5efce913/users/a8f2c9e2cb994438a6db6860a525b264/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:38.326507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8612bbf-a3e3-4e7d-8317-e59ff0b57522 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:38.453318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:38.489946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9c9da-5343-4bb4-a5b1-dd14d5b6d90f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:38.492339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9c9da-5343-4bb4-a5b1-dd14d5b6d90f service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:38.510090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 832/1647] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:38 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:38.520643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2490d-e4fe-4267-be7a-b39139665a0f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:38.520942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2490d-e4fe-4267-be7a-b39139665a0f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:38.521223 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2490d-e4fe-4267-be7a-b39139665a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:38.544736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8612bbf-a3e3-4e7d-8317-e59ff0b57522 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:38.564212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085e5a2-065e-46e6-ab47-802d8611d702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:38.564212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085e5a2-065e-46e6-ab47-802d8611d702 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:38.564212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085e5a2-065e-46e6-ab47-802d8611d702 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:38.564212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085e5a2-065e-46e6-ab47-802d8611d702 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:38.564675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085e5a2-065e-46e6-ab47-802d8611d702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:38.565799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085e5a2-065e-46e6-ab47-802d8611d702 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:38.596864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 816/1648] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:38 2022] GET /identity/v3/roles => generated 1427 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:38.599785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085e5a2-065e-46e6-ab47-802d8611d702 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:38.614314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b6e2490d-e4fe-4267-be7a-b39139665a0f None None] MFA Rules not processed for user `84b4d6be32e94ccd80f97aa8c93efbcc`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:38.615228 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b6e2490d-e4fe-4267-be7a-b39139665a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:38.616541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085e5a2-065e-46e6-ab47-802d8611d702 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:38.623055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec370502-f904-45f8-98c9-dd1c469727e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:38.624518 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec370502-f904-45f8-98c9-dd1c469727e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:38.624859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec370502-f904-45f8-98c9-dd1c469727e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:38.625241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec370502-f904-45f8-98c9-dd1c469727e9 admin admin] PATH_INFO: `/v3/projects/23b3bfaaf2974d51a6737b0e5efce913/users/a8f2c9e2cb994438a6db6860a525b264/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:38.628201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec370502-f904-45f8-98c9-dd1c469727e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b3bfaaf2974d51a6737b0e5efce913, user_id=a8f2c9e2cb994438a6db6860a525b264, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:38.631947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec370502-f904-45f8-98c9-dd1c469727e9 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:38.675395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 817/1649] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:38 2022] PUT /identity/v3/projects/23b3bfaaf2974d51a6737b0e5efce913/users/a8f2c9e2cb994438a6db6860a525b264/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:38.737917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:38.755161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 833/1650] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:38.768598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eda699-83d3-4fdf-a859-832462163c82 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:38.769409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eda699-83d3-4fdf-a859-832462163c82 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:38.769409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eda699-83d3-4fdf-a859-832462163c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:38.790150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-e3eda699-83d3-4fdf-a859-832462163c82 None None] MFA Rules not processed for user `84b4d6be32e94ccd80f97aa8c93efbcc`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:38.791735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e3eda699-83d3-4fdf-a859-832462163c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:38.867490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 818/1651] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:38.884371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82fe6b-6b4c-4003-8642-0ce398285d89 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:38.885839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82fe6b-6b4c-4003-8642-0ce398285d89 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:38.886592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82fe6b-6b4c-4003-8642-0ce398285d89 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:38.888159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eda699-83d3-4fdf-a859-832462163c82 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:38.889359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e3eda699-83d3-4fdf-a859-832462163c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:38.897763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 834/1652] 10.222.0.121 () {54 vars in 966 bytes} [Thu May 26 17:59:38 2022] GET /identity => generated 271 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:59:38.916915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c228905-a8e4-4212-ba03-e0d054efe5e4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:38.916915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c228905-a8e4-4212-ba03-e0d054efe5e4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:38.916915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c228905-a8e4-4212-ba03-e0d054efe5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:38.961277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7c228905-a8e4-4212-ba03-e0d054efe5e4 None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:38.963224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7c228905-a8e4-4212-ba03-e0d054efe5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:39.044292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eda699-83d3-4fdf-a859-832462163c82 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:39.066491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 835/1653] 10.222.0.121 () {58 vars in 1063 bytes} [Thu May 26 17:59:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:59:39.067645 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c228905-a8e4-4212-ba03-e0d054efe5e4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:39.069033 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7c228905-a8e4-4212-ba03-e0d054efe5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:39.070520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59c9c1-5811-497d-9d2d-eb7a17914cf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:39.071369 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59c9c1-5811-497d-9d2d-eb7a17914cf4 service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:39.071715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59c9c1-5811-497d-9d2d-eb7a17914cf4 service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:39.071939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59c9c1-5811-497d-9d2d-eb7a17914cf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:39.101134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59c9c1-5811-497d-9d2d-eb7a17914cf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:39.111387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59c9c1-5811-497d-9d2d-eb7a17914cf4 service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:39.134441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 819/1654] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 17:59:39.136081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b99f3-64d0-4592-bc9d-c7b5f7206b9d None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:39.136274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b99f3-64d0-4592-bc9d-c7b5f7206b9d None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:39.136543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b99f3-64d0-4592-bc9d-c7b5f7206b9d None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:39.141360 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 820/1655] 10.222.0.121 () {54 vars in 917 bytes} [Thu May 26 17:59:39 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:59:39.149277 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97576639-f24a-4971-a5b4-be556d3e68e9 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:39.149572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97576639-f24a-4971-a5b4-be556d3e68e9 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:39.149856 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97576639-f24a-4971-a5b4-be556d3e68e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:39.160627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c228905-a8e4-4212-ba03-e0d054efe5e4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:39.166605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:39.168111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:39.168569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:39.168929 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] PATH_INFO: `/v3/projects/18d58426facd4034a1e76fd7e52817b0` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:39.170312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18d58426facd4034a1e76fd7e52817b0)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:39.172159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 17:59:39.172159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: warnings.warn(msg) May 26 17:59:39.172159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: May 26 17:59:39.173677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:39.195560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-97576639-f24a-4971-a5b4-be556d3e68e9 None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:39.196719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-97576639-f24a-4971-a5b4-be556d3e68e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:39.219328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 821/1656] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 17:59:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:39.231800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97576639-f24a-4971-a5b4-be556d3e68e9 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:39.234181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-97576639-f24a-4971-a5b4-be556d3e68e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:39.377921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18d58426facd4034a1e76fd7e52817b0', 'request_id': 'req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:39.461386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97576639-f24a-4971-a5b4-be556d3e68e9 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:39.526005 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18d58426facd4034a1e76fd7e52817b0', 'request_id': 'req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:39.553813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72b1b4-ce1f-432b-a758-20fb954a5bef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:39.554663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72b1b4-ce1f-432b-a758-20fb954a5bef service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:39.555367 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72b1b4-ce1f-432b-a758-20fb954a5bef service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:39.555879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72b1b4-ce1f-432b-a758-20fb954a5bef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:39.556963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18d58426facd4034a1e76fd7e52817b0', 'request_id': 'req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:39.579941 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] The token cache is being invalidate because project 18d58426facd4034a1e76fd7e52817b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 17:59:39.580235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:39.583551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:39.587047 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 836/1657] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 17:59:39 2022] DELETE /identity/v3/projects/18d58426facd4034a1e76fd7e52817b0 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:39.591572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:39.597213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:39.720679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72b1b4-ce1f-432b-a758-20fb954a5bef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:39.723832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72b1b4-ce1f-432b-a758-20fb954a5bef service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:39.740530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 822/1658] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 17:59:39.747081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545de02-1f6b-4ff3-8eee-c344e8bb56c7 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:39.754202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a9bed-1d9b-47c7-97f5-f6bedcaabe69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:39.755250 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a9bed-1d9b-47c7-97f5-f6bedcaabe69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:39.755533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a9bed-1d9b-47c7-97f5-f6bedcaabe69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:39.756088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a9bed-1d9b-47c7-97f5-f6bedcaabe69 admin admin] PATH_INFO: `/v3/users/14afd27854224e758b4336515f628161` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:39.757191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a9bed-1d9b-47c7-97f5-f6bedcaabe69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14afd27854224e758b4336515f628161)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:39.759406 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a9bed-1d9b-47c7-97f5-f6bedcaabe69 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:40.194531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:40.245221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9f7a9bed-1d9b-47c7-97f5-f6bedcaabe69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14afd27854224e758b4336515f628161'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:40.260973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9f7a9bed-1d9b-47c7-97f5-f6bedcaabe69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14afd27854224e758b4336515f628161'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:40.263194 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9f7a9bed-1d9b-47c7-97f5-f6bedcaabe69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14afd27854224e758b4336515f628161'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:40.296968 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9f7a9bed-1d9b-47c7-97f5-f6bedcaabe69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14afd27854224e758b4336515f628161'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:40.381216 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9f7a9bed-1d9b-47c7-97f5-f6bedcaabe69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14afd27854224e758b4336515f628161'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:40.390438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 837/1659] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 17:59:39 2022] DELETE /identity/v3/users/14afd27854224e758b4336515f628161 => generated 0 bytes in 799 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:40.519594 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4671030-637c-4c27-b1cc-51f5be1e62f3 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:40.519883 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4671030-637c-4c27-b1cc-51f5be1e62f3 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:40.520209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4671030-637c-4c27-b1cc-51f5be1e62f3 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:40.523086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 823/1660] 10.222.0.121 () {54 vars in 917 bytes} [Thu May 26 17:59:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:59:40.542153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2c0af-7f15-489e-ad6f-600ddf081490 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:40.542451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2c0af-7f15-489e-ad6f-600ddf081490 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:40.542719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2c0af-7f15-489e-ad6f-600ddf081490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:40.643495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6db2c0af-7f15-489e-ad6f-600ddf081490 None None] MFA Rules not processed for user `08709b462300437598c8db331e5ad20f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:40.647246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6db2c0af-7f15-489e-ad6f-600ddf081490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:40.765043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 838/1661] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 17:59:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:40.946888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4671030-637c-4c27-b1cc-51f5be1e62f3 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:40.948861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b4671030-637c-4c27-b1cc-51f5be1e62f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:40.972859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4671030-637c-4c27-b1cc-51f5be1e62f3 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:40.982618 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc8e07-bf1c-4cbc-b618-f453a112efd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:40.983388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc8e07-bf1c-4cbc-b618-f453a112efd6 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:40.983871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc8e07-bf1c-4cbc-b618-f453a112efd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:40.984390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc8e07-bf1c-4cbc-b618-f453a112efd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:40.997912 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc8e07-bf1c-4cbc-b618-f453a112efd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:41.001662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc8e07-bf1c-4cbc-b618-f453a112efd6 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:41.035845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 824/1662] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:41.441220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:42.408043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:42.721532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2c0af-7f15-489e-ad6f-600ddf081490 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:42.723307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6db2c0af-7f15-489e-ad6f-600ddf081490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:42.874517 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7bfe6-1338-4a10-82d3-4267d5193c78 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:42.876310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7bfe6-1338-4a10-82d3-4267d5193c78 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:42.880406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7bfe6-1338-4a10-82d3-4267d5193c78 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:42.880406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 825/1663] 10.222.0.121 () {54 vars in 965 bytes} [Thu May 26 17:59:42 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 17:59:42.885514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123510f-b336-42de-929f-ed0f2ebf003e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:42.885798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123510f-b336-42de-929f-ed0f2ebf003e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:42.886399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123510f-b336-42de-929f-ed0f2ebf003e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:42.890771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2c0af-7f15-489e-ad6f-600ddf081490 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:42.909218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00165b1d-275e-4496-852d-e34e8c8e5677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:42.911501 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00165b1d-275e-4496-852d-e34e8c8e5677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:42.913094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00165b1d-275e-4496-852d-e34e8c8e5677 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:42.913590 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00165b1d-275e-4496-852d-e34e8c8e5677 admin admin] PATH_INFO: `/v3/projects/c1e2a3642be04d908265413df099b8f1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:42.917574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00165b1d-275e-4496-852d-e34e8c8e5677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e2a3642be04d908265413df099b8f1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:42.939057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00165b1d-275e-4496-852d-e34e8c8e5677 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:42.947776 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-4123510f-b336-42de-929f-ed0f2ebf003e None None] MFA Rules not processed for user `7847d2c119474d49bef6029d51dbefdb`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:42.955353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4123510f-b336-42de-929f-ed0f2ebf003e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:43.156264 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 826/1664] 10.222.0.121 () {58 vars in 1062 bytes} [Thu May 26 17:59:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:43.164889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123510f-b336-42de-929f-ed0f2ebf003e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:43.178075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123510f-b336-42de-929f-ed0f2ebf003e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:43.182769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-00165b1d-275e-4496-852d-e34e8c8e5677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e2a3642be04d908265413df099b8f1', 'request_id': 'req-00165b1d-275e-4496-852d-e34e8c8e5677'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:43.219609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:43.269295 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-00165b1d-275e-4496-852d-e34e8c8e5677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e2a3642be04d908265413df099b8f1', 'request_id': 'req-00165b1d-275e-4496-852d-e34e8c8e5677'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:43.279919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-00165b1d-275e-4496-852d-e34e8c8e5677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e2a3642be04d908265413df099b8f1', 'request_id': 'req-00165b1d-275e-4496-852d-e34e8c8e5677'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:43.323374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-00165b1d-275e-4496-852d-e34e8c8e5677 admin admin] The token cache is being invalidate because project c1e2a3642be04d908265413df099b8f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 17:59:43.323671 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-00165b1d-275e-4496-852d-e34e8c8e5677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:43.325959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-00165b1d-275e-4496-852d-e34e8c8e5677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:43.335901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 839/1665] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 17:59:42 2022] DELETE /identity/v3/projects/c1e2a3642be04d908265413df099b8f1 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:43.339650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef1081-8ac7-45ea-8c03-9c81a837346b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:43.339989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef1081-8ac7-45ea-8c03-9c81a837346b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:43.340272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef1081-8ac7-45ea-8c03-9c81a837346b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:43.411992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50c79b-10e2-4b57-8247-dfa9c4b81f2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:43.412698 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50c79b-10e2-4b57-8247-dfa9c4b81f2d service placement] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:43.412983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50c79b-10e2-4b57-8247-dfa9c4b81f2d service placement] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:43.416540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50c79b-10e2-4b57-8247-dfa9c4b81f2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:43.427863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3f50c79b-10e2-4b57-8247-dfa9c4b81f2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:43.432105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-88ef1081-8ac7-45ea-8c03-9c81a837346b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:43.440139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-88ef1081-8ac7-45ea-8c03-9c81a837346b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:43.470738 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50c79b-10e2-4b57-8247-dfa9c4b81f2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:43.487386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50c79b-10e2-4b57-8247-dfa9c4b81f2d service placement] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:43.544516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 827/1666] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 17:59:43.620305 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 840/1667] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:43.628515 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b1b64-5bac-43e5-92b2-31a6b8455d67 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:43.628999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b1b64-5bac-43e5-92b2-31a6b8455d67 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:43.635081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b1b64-5bac-43e5-92b2-31a6b8455d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:43.700796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-7a8b1b64-5bac-43e5-92b2-31a6b8455d67 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:43.710055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7a8b1b64-5bac-43e5-92b2-31a6b8455d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:43.741502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef1081-8ac7-45ea-8c03-9c81a837346b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:43.751942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 828/1668] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:59:43.760302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b114d-24ce-4cde-b7e2-44f23e7c6d4e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:43.761064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef1081-8ac7-45ea-8c03-9c81a837346b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:43.761915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b114d-24ce-4cde-b7e2-44f23e7c6d4e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:43.762861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b114d-24ce-4cde-b7e2-44f23e7c6d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:43.781691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451b11f-48d9-4ba9-a350-b80f77fad896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:43.786543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451b11f-48d9-4ba9-a350-b80f77fad896 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:43.787828 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451b11f-48d9-4ba9-a350-b80f77fad896 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:43.788123 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451b11f-48d9-4ba9-a350-b80f77fad896 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:43.789982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451b11f-48d9-4ba9-a350-b80f77fad896 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:59:43.792566 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451b11f-48d9-4ba9-a350-b80f77fad896 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:43.796238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451b11f-48d9-4ba9-a350-b80f77fad896 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:43.847819 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 841/1669] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 17:59:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:59:43.848916 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451b11f-48d9-4ba9-a350-b80f77fad896 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:43.862052 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-222b114d-24ce-4cde-b7e2-44f23e7c6d4e None None] MFA Rules not processed for user `a8f2c9e2cb994438a6db6860a525b264`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:43.865245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-222b114d-24ce-4cde-b7e2-44f23e7c6d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:43.876742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451b11f-48d9-4ba9-a350-b80f77fad896 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:43.896687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce50b16-b08c-4c24-8ee6-d10d36d0914d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:43.898579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce50b16-b08c-4c24-8ee6-d10d36d0914d admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:43.902283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce50b16-b08c-4c24-8ee6-d10d36d0914d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:43.906399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce50b16-b08c-4c24-8ee6-d10d36d0914d admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:43.915096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce50b16-b08c-4c24-8ee6-d10d36d0914d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:43.917998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce50b16-b08c-4c24-8ee6-d10d36d0914d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:44.017345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 829/1670] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:43 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:44.149832 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 842/1671] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:43 2022] POST /identity/v3/projects => generated 368 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:44.154589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b114d-24ce-4cde-b7e2-44f23e7c6d4e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:44.174909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b114d-24ce-4cde-b7e2-44f23e7c6d4e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:44.309700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbdebb-979a-4aaa-a508-60d11ee74fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:44.310517 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbdebb-979a-4aaa-a508-60d11ee74fae admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:44.310970 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbdebb-979a-4aaa-a508-60d11ee74fae admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:44.311349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbdebb-979a-4aaa-a508-60d11ee74fae admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:44.312376 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbdebb-979a-4aaa-a508-60d11ee74fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:44.315068 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbdebb-979a-4aaa-a508-60d11ee74fae admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:44.413176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce50b16-b08c-4c24-8ee6-d10d36d0914d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:44.413340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 830/1672] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:44 2022] POST /identity/v3/users => generated 285 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:44.430550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce50b16-b08c-4c24-8ee6-d10d36d0914d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:44.447502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56289e4-08d2-4bfa-ac14-03281f38d1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:44.448222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56289e4-08d2-4bfa-ac14-03281f38d1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:44.460356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56289e4-08d2-4bfa-ac14-03281f38d1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:44.461664 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56289e4-08d2-4bfa-ac14-03281f38d1d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:44.463426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56289e4-08d2-4bfa-ac14-03281f38d1d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:44.465259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56289e4-08d2-4bfa-ac14-03281f38d1d4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:44.482166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 843/1673] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:44 2022] GET /identity/v3/roles => generated 1427 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:59:44.483138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbdebb-979a-4aaa-a508-60d11ee74fae admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:44.505062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbdebb-979a-4aaa-a508-60d11ee74fae admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:44.513844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e315d-a1cd-4710-8446-86cba5e14621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:44.514726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e315d-a1cd-4710-8446-86cba5e14621 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:44.515182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e315d-a1cd-4710-8446-86cba5e14621 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:44.515563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e315d-a1cd-4710-8446-86cba5e14621 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:44.516431 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e315d-a1cd-4710-8446-86cba5e14621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:44.518224 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e315d-a1cd-4710-8446-86cba5e14621 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:44.551950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 831/1674] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:44 2022] GET /identity/v3/roles => generated 1427 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:44.568423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56289e4-08d2-4bfa-ac14-03281f38d1d4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:44.581470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56289e4-08d2-4bfa-ac14-03281f38d1d4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:44.590835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f4b02-0148-4696-832b-66b610fafb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:44.591764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f4b02-0148-4696-832b-66b610fafb6e admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:44.592040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f4b02-0148-4696-832b-66b610fafb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:44.592309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f4b02-0148-4696-832b-66b610fafb6e admin admin] PATH_INFO: `/v3/projects/c769fd9e29fe49e083166beec17e9712/users/8085a2847e2d482f94117d5c6482b092/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:44.622183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f4b02-0148-4696-832b-66b610fafb6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c769fd9e29fe49e083166beec17e9712, user_id=8085a2847e2d482f94117d5c6482b092, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:44.625739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f4b02-0148-4696-832b-66b610fafb6e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:44.723949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 844/1675] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:44 2022] PUT /identity/v3/projects/c769fd9e29fe49e083166beec17e9712/users/8085a2847e2d482f94117d5c6482b092/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:44.742396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798defa-1e9c-47ae-83e3-ff172e50b86e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:44.742710 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798defa-1e9c-47ae-83e3-ff172e50b86e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:44.743397 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798defa-1e9c-47ae-83e3-ff172e50b86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:44.751102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:44.765308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:44.810907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-e798defa-1e9c-47ae-83e3-ff172e50b86e None None] MFA Rules not processed for user `8085a2847e2d482f94117d5c6482b092`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:44.813186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e798defa-1e9c-47ae-83e3-ff172e50b86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:44.994550 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 832/1676] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:44.996579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f4b02-0148-4696-832b-66b610fafb6e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:45.088947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f4b02-0148-4696-832b-66b610fafb6e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:45.096530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2c5e0-f6dc-490d-9779-2aac413fe81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:45.098193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2c5e0-f6dc-490d-9779-2aac413fe81f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:45.098616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2c5e0-f6dc-490d-9779-2aac413fe81f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:45.099343 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2c5e0-f6dc-490d-9779-2aac413fe81f admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:45.104929 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2c5e0-f6dc-490d-9779-2aac413fe81f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:45.107726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2c5e0-f6dc-490d-9779-2aac413fe81f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:45.246317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 845/1677] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:44 2022] POST /identity/v3/projects => generated 370 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:45.251198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798defa-1e9c-47ae-83e3-ff172e50b86e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:45.273306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798defa-1e9c-47ae-83e3-ff172e50b86e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:45.381205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32409859-7af7-4763-8993-419935adaedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:45.382401 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32409859-7af7-4763-8993-419935adaedc admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:45.382693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32409859-7af7-4763-8993-419935adaedc admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:45.383023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32409859-7af7-4763-8993-419935adaedc admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:45.383743 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32409859-7af7-4763-8993-419935adaedc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:45.388640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32409859-7af7-4763-8993-419935adaedc admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:45.543162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 833/1678] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:45 2022] POST /identity/v3/users => generated 292 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:59:45.547870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2c5e0-f6dc-490d-9779-2aac413fe81f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:45.565525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2c5e0-f6dc-490d-9779-2aac413fe81f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:45.583422 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af828733-4145-4ddb-bec6-9a3579bd6fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:45.584178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af828733-4145-4ddb-bec6-9a3579bd6fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:45.584468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af828733-4145-4ddb-bec6-9a3579bd6fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:45.584732 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af828733-4145-4ddb-bec6-9a3579bd6fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:45.585470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af828733-4145-4ddb-bec6-9a3579bd6fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:45.587116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af828733-4145-4ddb-bec6-9a3579bd6fb2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:45.617791 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 846/1679] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:45 2022] GET /identity/v3/roles => generated 1427 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:59:45.622236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32409859-7af7-4763-8993-419935adaedc admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:45.635826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32409859-7af7-4763-8993-419935adaedc admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:45.642975 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6471721-5183-4869-8078-e4a9c9a10a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:45.643603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6471721-5183-4869-8078-e4a9c9a10a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:45.643885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6471721-5183-4869-8078-e4a9c9a10a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:45.644354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6471721-5183-4869-8078-e4a9c9a10a9d admin admin] PATH_INFO: `/v3/projects/040599db0d2947328ab558945c562ae4/users/37d12e7a96f34d9c80e0ac2c7027c7ea/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:45.705255 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6471721-5183-4869-8078-e4a9c9a10a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=040599db0d2947328ab558945c562ae4, user_id=37d12e7a96f34d9c80e0ac2c7027c7ea, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:45.708147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6471721-5183-4869-8078-e4a9c9a10a9d admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:45.841979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 834/1680] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:45 2022] PUT /identity/v3/projects/040599db0d2947328ab558945c562ae4/users/37d12e7a96f34d9c80e0ac2c7027c7ea/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:45.846720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af828733-4145-4ddb-bec6-9a3579bd6fb2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:45.858158 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af828733-4145-4ddb-bec6-9a3579bd6fb2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:45.960988 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20536f-ea1d-4860-8e68-a614ded2a753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:45.961748 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20536f-ea1d-4860-8e68-a614ded2a753 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:45.962095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20536f-ea1d-4860-8e68-a614ded2a753 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:45.962440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20536f-ea1d-4860-8e68-a614ded2a753 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:45.963465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20536f-ea1d-4860-8e68-a614ded2a753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:45.965624 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20536f-ea1d-4860-8e68-a614ded2a753 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:45.989213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 847/1681] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:45 2022] GET /identity/v3/roles => generated 1427 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:59:45.994430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6471721-5183-4869-8078-e4a9c9a10a9d admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:46.021424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6471721-5183-4869-8078-e4a9c9a10a9d admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:46.031583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea739ea-4c1a-4393-8c9a-c4f0f9e2e32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:46.033141 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea739ea-4c1a-4393-8c9a-c4f0f9e2e32d admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:46.033141 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea739ea-4c1a-4393-8c9a-c4f0f9e2e32d admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:46.033141 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea739ea-4c1a-4393-8c9a-c4f0f9e2e32d admin admin] PATH_INFO: `/v3/projects/040599db0d2947328ab558945c562ae4/users/37d12e7a96f34d9c80e0ac2c7027c7ea/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:46.035369 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea739ea-4c1a-4393-8c9a-c4f0f9e2e32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=040599db0d2947328ab558945c562ae4, user_id=37d12e7a96f34d9c80e0ac2c7027c7ea, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:46.038688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea739ea-4c1a-4393-8c9a-c4f0f9e2e32d admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:46.055240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 835/1682] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:45 2022] PUT /identity/v3/projects/040599db0d2947328ab558945c562ae4/users/37d12e7a96f34d9c80e0ac2c7027c7ea/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:46.069438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae64c1-aa3d-4f7e-bee2-93ac9a716b1e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:46.071959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae64c1-aa3d-4f7e-bee2-93ac9a716b1e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:46.073304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae64c1-aa3d-4f7e-bee2-93ac9a716b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:46.161660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-65ae64c1-aa3d-4f7e-bee2-93ac9a716b1e None None] MFA Rules not processed for user `37d12e7a96f34d9c80e0ac2c7027c7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:46.166690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-65ae64c1-aa3d-4f7e-bee2-93ac9a716b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:46.334097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 848/1683] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:46.347109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea739ea-4c1a-4393-8c9a-c4f0f9e2e32d admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:46.349032 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1ea739ea-4c1a-4393-8c9a-c4f0f9e2e32d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:46.428924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea739ea-4c1a-4393-8c9a-c4f0f9e2e32d admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:46.433595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0580b7-6d22-42b1-824e-0d85eedebb97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:46.435786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0580b7-6d22-42b1-824e-0d85eedebb97 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:46.436304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0580b7-6d22-42b1-824e-0d85eedebb97 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:46.436882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0580b7-6d22-42b1-824e-0d85eedebb97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:46.454558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0580b7-6d22-42b1-824e-0d85eedebb97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:46.456642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0580b7-6d22-42b1-824e-0d85eedebb97 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:46.475614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238241ee-8d93-4fd4-84e0-704c3e03b116 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:46.476120 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238241ee-8d93-4fd4-84e0-704c3e03b116 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:46.476534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238241ee-8d93-4fd4-84e0-704c3e03b116 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:46.479637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 849/1684] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 17:59:46 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:59:46.490991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 836/1685] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:46.505545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271c834-b2fb-4af3-a23e-333cbd9e58d3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:46.506043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271c834-b2fb-4af3-a23e-333cbd9e58d3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:46.506285 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271c834-b2fb-4af3-a23e-333cbd9e58d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:46.528681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1271c834-b2fb-4af3-a23e-333cbd9e58d3 None None] MFA Rules not processed for user `7847d2c119474d49bef6029d51dbefdb`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:46.530742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1271c834-b2fb-4af3-a23e-333cbd9e58d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:46.633674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 850/1686] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 17:59:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:46.687556 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:46.689091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0580b7-6d22-42b1-824e-0d85eedebb97 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:46.690786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ca0580b7-6d22-42b1-824e-0d85eedebb97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:46.750874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0580b7-6d22-42b1-824e-0d85eedebb97 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:46.775020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d931f5-9c00-4c85-a6d7-cbd1d0374a89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:46.775701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d931f5-9c00-4c85-a6d7-cbd1d0374a89 service placement] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:46.776706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d931f5-9c00-4c85-a6d7-cbd1d0374a89 service placement] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:46.777553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d931f5-9c00-4c85-a6d7-cbd1d0374a89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:46.801825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d931f5-9c00-4c85-a6d7-cbd1d0374a89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:46.804408 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d931f5-9c00-4c85-a6d7-cbd1d0374a89 service placement] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:46.841070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 837/1687] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:47.428303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:47.520816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f03ee-bb4d-4023-9d53-b1fdf735e9a8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:47.521242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f03ee-bb4d-4023-9d53-b1fdf735e9a8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:47.521515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f03ee-bb4d-4023-9d53-b1fdf735e9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:47.634161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9a1f03ee-bb4d-4023-9d53-b1fdf735e9a8 None None] MFA Rules not processed for user `dbee2da50ee6413db771071900703e29`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:47.642243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9a1f03ee-bb4d-4023-9d53-b1fdf735e9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:47.845405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 851/1688] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:59:47.847271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259724f0-5661-4cec-9b6f-a698f1c8636f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:47.847743 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259724f0-5661-4cec-9b6f-a698f1c8636f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:47.847949 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259724f0-5661-4cec-9b6f-a698f1c8636f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:47.875012 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f03ee-bb4d-4023-9d53-b1fdf735e9a8 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:47.900934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-259724f0-5661-4cec-9b6f-a698f1c8636f None None] MFA Rules not processed for user `dbee2da50ee6413db771071900703e29`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:47.905024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-259724f0-5661-4cec-9b6f-a698f1c8636f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:47.941162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 838/1689] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:47.956711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259724f0-5661-4cec-9b6f-a698f1c8636f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:47.958456 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-259724f0-5661-4cec-9b6f-a698f1c8636f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:48.140403 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f03ee-bb4d-4023-9d53-b1fdf735e9a8 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:48.147139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:48.148401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:48.149324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:48.149745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] PATH_INFO: `/v3/users/8085a2847e2d482f94117d5c6482b092` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:48.172380 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8085a2847e2d482f94117d5c6482b092)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:48.178528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:48.221316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259724f0-5661-4cec-9b6f-a698f1c8636f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:48.232412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3672a6-5ebd-4fc8-945d-82150f7ee1a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:48.233230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3672a6-5ebd-4fc8-945d-82150f7ee1a5 service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:48.235568 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3672a6-5ebd-4fc8-945d-82150f7ee1a5 service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:48.235568 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3672a6-5ebd-4fc8-945d-82150f7ee1a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:48.259386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3672a6-5ebd-4fc8-945d-82150f7ee1a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:48.263211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3672a6-5ebd-4fc8-945d-82150f7ee1a5 service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:48.295247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 839/1690] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 17:59:48.377307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d2cda-7f24-46b8-8e20-8c93aad7e1da None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:48.377593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d2cda-7f24-46b8-8e20-8c93aad7e1da None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:48.377875 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d2cda-7f24-46b8-8e20-8c93aad7e1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:48.464231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-004d2cda-7f24-46b8-8e20-8c93aad7e1da None None] MFA Rules not processed for user `fe1c86c411834f1c8d2775a5b2c5f1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:48.465536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-004d2cda-7f24-46b8-8e20-8c93aad7e1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:48.544058 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8085a2847e2d482f94117d5c6482b092'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:48.568704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8085a2847e2d482f94117d5c6482b092'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:48.572452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8085a2847e2d482f94117d5c6482b092'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:48.612768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8085a2847e2d482f94117d5c6482b092'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:48.682583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 840/1691] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:59:48.696334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8085a2847e2d482f94117d5c6482b092'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:48.709850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 852/1692] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 17:59:47 2022] DELETE /identity/v3/users/8085a2847e2d482f94117d5c6482b092 => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:48.712844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752d6cd-b084-4eec-82cc-cfcab0a9ca12 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:48.712844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752d6cd-b084-4eec-82cc-cfcab0a9ca12 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:48.712844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752d6cd-b084-4eec-82cc-cfcab0a9ca12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:48.772326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f752d6cd-b084-4eec-82cc-cfcab0a9ca12 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:48.772963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:48.773619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f752d6cd-b084-4eec-82cc-cfcab0a9ca12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:48.774922 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:48.886806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:48.934351 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 841/1693] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:48.944575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752d6cd-b084-4eec-82cc-cfcab0a9ca12 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:48.946272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f752d6cd-b084-4eec-82cc-cfcab0a9ca12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:48.965088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c6e01-5d0f-4987-92a3-bbd3a9d787b0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:48.982037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fdc21-ba29-4c91-a1fa-b33a9650db08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:48.982844 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fdc21-ba29-4c91-a1fa-b33a9650db08 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:48.983202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fdc21-ba29-4c91-a1fa-b33a9650db08 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:48.983688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fdc21-ba29-4c91-a1fa-b33a9650db08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:48.991286 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-512fdc21-ba29-4c91-a1fa-b33a9650db08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:49.064687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752d6cd-b084-4eec-82cc-cfcab0a9ca12 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:49.075561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dfad6-0aab-4357-b9a7-957ad4284fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:49.076692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dfad6-0aab-4357-b9a7-957ad4284fcc service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:49.076692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dfad6-0aab-4357-b9a7-957ad4284fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:49.076943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dfad6-0aab-4357-b9a7-957ad4284fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:49.120958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dfad6-0aab-4357-b9a7-957ad4284fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:49.137373 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dfad6-0aab-4357-b9a7-957ad4284fcc service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:49.201213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 842/1694] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:49.214196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fdc21-ba29-4c91-a1fa-b33a9650db08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:49.216906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fdc21-ba29-4c91-a1fa-b33a9650db08 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:49.236223 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 853/1695] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:49.259016 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:49.487800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500b6bd-6f41-4a7d-ac53-c06e9eee03c1 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:49.488272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500b6bd-6f41-4a7d-ac53-c06e9eee03c1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:49.489882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500b6bd-6f41-4a7d-ac53-c06e9eee03c1 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:49.491994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 843/1696] 10.222.0.121 () {54 vars in 917 bytes} [Thu May 26 17:59:49 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 17:59:49.506708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6b6cf-eeb2-4054-ada6-f28627ba935d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:49.507149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6b6cf-eeb2-4054-ada6-f28627ba935d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:49.507495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6b6cf-eeb2-4054-ada6-f28627ba935d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:49.551589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b9a6b6cf-eeb2-4054-ada6-f28627ba935d None None] MFA Rules not processed for user `08709b462300437598c8db331e5ad20f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:49.554992 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b9a6b6cf-eeb2-4054-ada6-f28627ba935d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:49.603719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 854/1697] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 17:59:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:49.704679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500b6bd-6f41-4a7d-ac53-c06e9eee03c1 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:49.707260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6500b6bd-6f41-4a7d-ac53-c06e9eee03c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:49.724912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6b6cf-eeb2-4054-ada6-f28627ba935d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:49.727198 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b9a6b6cf-eeb2-4054-ada6-f28627ba935d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:49.750239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6b6cf-eeb2-4054-ada6-f28627ba935d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:49.760147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500b6bd-6f41-4a7d-ac53-c06e9eee03c1 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:49.769783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253e1c6-e92a-406e-8a36-24abb58cca6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:49.778062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253e1c6-e92a-406e-8a36-24abb58cca6c service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:49.778351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253e1c6-e92a-406e-8a36-24abb58cca6c service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:49.778723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253e1c6-e92a-406e-8a36-24abb58cca6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:49.787850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde15de9-d4d7-46da-865c-108af2b9f9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:49.788746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde15de9-d4d7-46da-865c-108af2b9f9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:49.789028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde15de9-d4d7-46da-865c-108af2b9f9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:49.789305 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde15de9-d4d7-46da-865c-108af2b9f9ed admin admin] PATH_INFO: `/v3/users/bbe7dec7280541cf98c3ab6f56b06c7e` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:49.805994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde15de9-d4d7-46da-865c-108af2b9f9ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe7dec7280541cf98c3ab6f56b06c7e)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:49.810562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: WARNING py.warnings [None req-bde15de9-d4d7-46da-865c-108af2b9f9ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 17:59:49.810562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: warnings.warn(msg) May 26 17:59:49.810562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: May 26 17:59:49.818999 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253e1c6-e92a-406e-8a36-24abb58cca6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:49.819339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde15de9-d4d7-46da-865c-108af2b9f9ed admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:49.823426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253e1c6-e92a-406e-8a36-24abb58cca6c service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:49.855323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 855/1698] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:50.058807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:50.137580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eb15e-3827-4ac5-a6f2-5e8a9d7b00c6 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:50.137866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eb15e-3827-4ac5-a6f2-5e8a9d7b00c6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:50.138140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eb15e-3827-4ac5-a6f2-5e8a9d7b00c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:50.194121 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bde15de9-d4d7-46da-865c-108af2b9f9ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe7dec7280541cf98c3ab6f56b06c7e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:50.196744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3e9eb15e-3827-4ac5-a6f2-5e8a9d7b00c6 None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:50.204500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3e9eb15e-3827-4ac5-a6f2-5e8a9d7b00c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:50.216261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bde15de9-d4d7-46da-865c-108af2b9f9ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe7dec7280541cf98c3ab6f56b06c7e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:50.217030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bde15de9-d4d7-46da-865c-108af2b9f9ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe7dec7280541cf98c3ab6f56b06c7e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:50.290419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bde15de9-d4d7-46da-865c-108af2b9f9ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe7dec7280541cf98c3ab6f56b06c7e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:50.365023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 856/1699] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 17:59:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:59:50.414658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bde15de9-d4d7-46da-865c-108af2b9f9ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe7dec7280541cf98c3ab6f56b06c7e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:50.430254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 844/1700] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 17:59:49 2022] DELETE /identity/v3/users/bbe7dec7280541cf98c3ab6f56b06c7e => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:50.797826 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eb15e-3827-4ac5-a6f2-5e8a9d7b00c6 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:50.800254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3e9eb15e-3827-4ac5-a6f2-5e8a9d7b00c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:50.957861 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eb15e-3827-4ac5-a6f2-5e8a9d7b00c6 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:51.062474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d8fc0-c26f-4c51-9b8d-e476ad1c44c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:51.063890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d8fc0-c26f-4c51-9b8d-e476ad1c44c0 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:51.064552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d8fc0-c26f-4c51-9b8d-e476ad1c44c0 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:51.065647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d8fc0-c26f-4c51-9b8d-e476ad1c44c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:51.070509 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f16d8fc0-c26f-4c51-9b8d-e476ad1c44c0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:51.260771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d8fc0-c26f-4c51-9b8d-e476ad1c44c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:51.263341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d8fc0-c26f-4c51-9b8d-e476ad1c44c0 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:51.297603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 857/1701] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 17:59:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 17:59:51.650959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:51.764343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862df2b-4868-485c-aa76-480eb250857a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:51.765501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862df2b-4868-485c-aa76-480eb250857a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:51.765941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862df2b-4868-485c-aa76-480eb250857a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:51.829524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b862df2b-4868-485c-aa76-480eb250857a None None] MFA Rules not processed for user `dbee2da50ee6413db771071900703e29`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:51.834795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b862df2b-4868-485c-aa76-480eb250857a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:52.091644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 845/1702] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:52.197634 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d8fc0-c26f-4c51-9b8d-e476ad1c44c0 service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:52.199443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f16d8fc0-c26f-4c51-9b8d-e476ad1c44c0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:52.505293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d8fc0-c26f-4c51-9b8d-e476ad1c44c0 service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:52.518395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:52.521183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:52.522048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:52.522562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525 admin admin] PATH_INFO: `/v3/projects/850db3538ad34024a004f5d3cea7d64e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:52.525468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=850db3538ad34024a004f5d3cea7d64e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:52.527482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:52.802380 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '850db3538ad34024a004f5d3cea7d64e', 'request_id': 'req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:53.262202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:53.393346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '850db3538ad34024a004f5d3cea7d64e', 'request_id': 'req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:53.415858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '850db3538ad34024a004f5d3cea7d64e', 'request_id': 'req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:53.432376 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862df2b-4868-485c-aa76-480eb250857a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:53.434211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b862df2b-4868-485c-aa76-480eb250857a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:53.483128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525 admin admin] The token cache is being invalidate because project 850db3538ad34024a004f5d3cea7d64e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 17:59:53.484286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862df2b-4868-485c-aa76-480eb250857a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:53.484607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:53.485690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f9128602-8bcb-4fb5-a5e8-ad0d7de48525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:53.495804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 858/1703] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 17:59:52 2022] DELETE /identity/v3/projects/850db3538ad34024a004f5d3cea7d64e => generated 0 bytes in 1296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:53.498926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ef6a-4a44-457e-837a-221b1929d1a7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:53.502096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:53.505824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ef6a-4a44-457e-837a-221b1929d1a7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:53.506337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ef6a-4a44-457e-837a-221b1929d1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:53.570944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1259ef6a-4a44-457e-837a-221b1929d1a7 None None] MFA Rules not processed for user `08709b462300437598c8db331e5ad20f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:53.580638 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1259ef6a-4a44-457e-837a-221b1929d1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:53.713079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 859/1704] 10.222.0.129 () {58 vars in 1020 bytes} [Thu May 26 17:59:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:53.732347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259ef6a-4a44-457e-837a-221b1929d1a7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:53.732347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1259ef6a-4a44-457e-837a-221b1929d1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:53.817794 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:53.818705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:53.819054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:53.819333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] PATH_INFO: `/v3/projects/c769fd9e29fe49e083166beec17e9712` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:53.820631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c769fd9e29fe49e083166beec17e9712)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:53.827059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:53.980862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259ef6a-4a44-457e-837a-221b1929d1a7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:53.987363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:53.988302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:53.988582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:53.988854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] PATH_INFO: `/v3/users/a8f2c9e2cb994438a6db6860a525b264` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:53.990114 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f2c9e2cb994438a6db6860a525b264)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:53.992898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:54.107671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c769fd9e29fe49e083166beec17e9712', 'request_id': 'req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:54.212546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c769fd9e29fe49e083166beec17e9712', 'request_id': 'req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:54.220445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c769fd9e29fe49e083166beec17e9712', 'request_id': 'req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:54.279106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] The token cache is being invalidate because project c769fd9e29fe49e083166beec17e9712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 17:59:54.279579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:54.282512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:54.292961 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 846/1705] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 17:59:53 2022] DELETE /identity/v3/projects/c769fd9e29fe49e083166beec17e9712 => generated 0 bytes in 861 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:59:54.293902 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:54.316184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:54.330855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:54.372444 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f2c9e2cb994438a6db6860a525b264'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:54.416289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f2c9e2cb994438a6db6860a525b264'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:54.417956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f2c9e2cb994438a6db6860a525b264'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:54.470736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f2c9e2cb994438a6db6860a525b264'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:54.558969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f2c9e2cb994438a6db6860a525b264'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:54.559842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58a5bf-8d8c-4f40-a066-e3f51d6481a1 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:54.569493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 860/1706] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 17:59:53 2022] DELETE /identity/v3/users/a8f2c9e2cb994438a6db6860a525b264 => generated 0 bytes in 856 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:54.573899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:54.574215 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b22571-7f89-4ee4-b834-ea32a388d084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:54.574725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b22571-7f89-4ee4-b834-ea32a388d084 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:54.575133 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b22571-7f89-4ee4-b834-ea32a388d084 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:54.575678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b22571-7f89-4ee4-b834-ea32a388d084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:54.577824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:54.580428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-00b22571-7f89-4ee4-b834-ea32a388d084 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:54.621023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b22571-7f89-4ee4-b834-ea32a388d084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:54.625480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b22571-7f89-4ee4-b834-ea32a388d084 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:54.659047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 847/1707] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:54.757441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b22571-7f89-4ee4-b834-ea32a388d084 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:54.759081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-00b22571-7f89-4ee4-b834-ea32a388d084 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:54.927745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eddac6-f213-4931-95e9-3518abceaa69 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:54.948128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b185e3c-15d8-4471-bd3e-df62b97f7916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:54.951271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b185e3c-15d8-4471-bd3e-df62b97f7916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:54.951944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b185e3c-15d8-4471-bd3e-df62b97f7916 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:54.952520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b185e3c-15d8-4471-bd3e-df62b97f7916 admin admin] PATH_INFO: `/v3/users/37d12e7a96f34d9c80e0ac2c7027c7ea` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:54.959837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b185e3c-15d8-4471-bd3e-df62b97f7916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37d12e7a96f34d9c80e0ac2c7027c7ea)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:54.964730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b185e3c-15d8-4471-bd3e-df62b97f7916 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:55.055589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b22571-7f89-4ee4-b834-ea32a388d084 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:55.082999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74237a5b-abb3-4571-83ec-206219ab8241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:55.086340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74237a5b-abb3-4571-83ec-206219ab8241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:55.087168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74237a5b-abb3-4571-83ec-206219ab8241 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:55.089950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74237a5b-abb3-4571-83ec-206219ab8241 admin admin] PATH_INFO: `/v3/users/146896e42498467b9b42cb49d73db2e8` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:55.097249 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74237a5b-abb3-4571-83ec-206219ab8241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=146896e42498467b9b42cb49d73db2e8)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:55.099507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74237a5b-abb3-4571-83ec-206219ab8241 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:55.284903 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3b185e3c-15d8-4471-bd3e-df62b97f7916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37d12e7a96f34d9c80e0ac2c7027c7ea'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:55.322353 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3b185e3c-15d8-4471-bd3e-df62b97f7916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37d12e7a96f34d9c80e0ac2c7027c7ea'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:55.323488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3b185e3c-15d8-4471-bd3e-df62b97f7916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37d12e7a96f34d9c80e0ac2c7027c7ea'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:55.354653 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3b185e3c-15d8-4471-bd3e-df62b97f7916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37d12e7a96f34d9c80e0ac2c7027c7ea'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:55.469720 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-74237a5b-abb3-4571-83ec-206219ab8241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '146896e42498467b9b42cb49d73db2e8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:55.482263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3b185e3c-15d8-4471-bd3e-df62b97f7916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37d12e7a96f34d9c80e0ac2c7027c7ea'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:55.489686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 861/1708] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 17:59:54 2022] DELETE /identity/v3/users/37d12e7a96f34d9c80e0ac2c7027c7ea => generated 0 bytes in 920 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:55.502126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-74237a5b-abb3-4571-83ec-206219ab8241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '146896e42498467b9b42cb49d73db2e8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:55.503574 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-74237a5b-abb3-4571-83ec-206219ab8241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '146896e42498467b9b42cb49d73db2e8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:55.525411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-74237a5b-abb3-4571-83ec-206219ab8241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '146896e42498467b9b42cb49d73db2e8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:55.698851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-74237a5b-abb3-4571-83ec-206219ab8241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '146896e42498467b9b42cb49d73db2e8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:55.709125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 848/1709] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 17:59:54 2022] DELETE /identity/v3/users/146896e42498467b9b42cb49d73db2e8 => generated 0 bytes in 952 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:59:55.714149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c723be-0ca9-40ff-8af6-610bdb8fb942 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:55.714433 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c723be-0ca9-40ff-8af6-610bdb8fb942 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:55.714952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c723be-0ca9-40ff-8af6-610bdb8fb942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:55.797138 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-70c723be-0ca9-40ff-8af6-610bdb8fb942 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:55.801722 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-70c723be-0ca9-40ff-8af6-610bdb8fb942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:56.016059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9846d7-6bb7-4759-9051-dce2046d5eb6 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:56.016544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9846d7-6bb7-4759-9051-dce2046d5eb6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:56.018231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9846d7-6bb7-4759-9051-dce2046d5eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:56.082515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 862/1710] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:56.112510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-2d9846d7-6bb7-4759-9051-dce2046d5eb6 None None] MFA Rules not processed for user `4ffcb99923b346938eb72c0b2786e815`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:56.113782 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-2d9846d7-6bb7-4759-9051-dce2046d5eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:56.177207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c723be-0ca9-40ff-8af6-610bdb8fb942 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:56.182633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-70c723be-0ca9-40ff-8af6-610bdb8fb942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:56.327989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c723be-0ca9-40ff-8af6-610bdb8fb942 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:56.338570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37a818-4f4a-4a39-91ac-ba12e8bd10fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:56.343964 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37a818-4f4a-4a39-91ac-ba12e8bd10fa service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:56.344357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37a818-4f4a-4a39-91ac-ba12e8bd10fa service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:56.344641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37a818-4f4a-4a39-91ac-ba12e8bd10fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:56.350319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:56.391802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37a818-4f4a-4a39-91ac-ba12e8bd10fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:56.391802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37a818-4f4a-4a39-91ac-ba12e8bd10fa service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:56.415728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 849/1711] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:56.433650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9846d7-6bb7-4759-9051-dce2046d5eb6 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:56.443881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-2d9846d7-6bb7-4759-9051-dce2046d5eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:56.473686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 863/1712] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:56.578537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9846d7-6bb7-4759-9051-dce2046d5eb6 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:56.586233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0443516-1518-4ede-8efb-b6207adcfe7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:56.587799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0443516-1518-4ede-8efb-b6207adcfe7e service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:56.588538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0443516-1518-4ede-8efb-b6207adcfe7e service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:56.589053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0443516-1518-4ede-8efb-b6207adcfe7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:56.600839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0443516-1518-4ede-8efb-b6207adcfe7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:56.603800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0443516-1518-4ede-8efb-b6207adcfe7e service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:56.618065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 850/1713] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 17:59:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 17:59:56.887441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37a818-4f4a-4a39-91ac-ba12e8bd10fa service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:56.893135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8e37a818-4f4a-4a39-91ac-ba12e8bd10fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:56.927373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37a818-4f4a-4a39-91ac-ba12e8bd10fa service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:56.947542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:56.948625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:56.948940 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:56.951364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] PATH_INFO: `/v3/projects/040599db0d2947328ab558945c562ae4` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:56.954095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] RBAC: Authorizing `identity:delete_project(project_id=040599db0d2947328ab558945c562ae4)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:56.960121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:56.966221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:57.137882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '040599db0d2947328ab558945c562ae4', 'request_id': 'req-726668b9-434f-4769-af36-ebb55e1ab62c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:57.251395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '040599db0d2947328ab558945c562ae4', 'request_id': 'req-726668b9-434f-4769-af36-ebb55e1ab62c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:57.263656 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '040599db0d2947328ab558945c562ae4', 'request_id': 'req-726668b9-434f-4769-af36-ebb55e1ab62c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:57.319249 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] The token cache is being invalidate because project 040599db0d2947328ab558945c562ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 17:59:57.320406 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:57.326627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:57.329871 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 864/1714] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 17:59:56 2022] DELETE /identity/v3/projects/040599db0d2947328ab558945c562ae4 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:57.336107 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31297c09-6c48-421c-a2b2-c98051688346 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:57.336650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31297c09-6c48-421c-a2b2-c98051688346 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:57.336650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31297c09-6c48-421c-a2b2-c98051688346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:57.391529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-31297c09-6c48-421c-a2b2-c98051688346 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:57.399158 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-31297c09-6c48-421c-a2b2-c98051688346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:57.412850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:57.414471 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:57.627496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 851/1715] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:57.639716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7aac7-2148-4161-af97-6d37dbaa2b83 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:57.643125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7aac7-2148-4161-af97-6d37dbaa2b83 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:57.643816 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7aac7-2148-4161-af97-6d37dbaa2b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:57.648821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726668b9-434f-4769-af36-ebb55e1ab62c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:57.689354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962ad4a-2202-4afa-9460-3d67770ade20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:57.705005 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962ad4a-2202-4afa-9460-3d67770ade20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:57.706405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962ad4a-2202-4afa-9460-3d67770ade20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:57.715298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962ad4a-2202-4afa-9460-3d67770ade20 admin admin] PATH_INFO: `/v3/projects/23b3bfaaf2974d51a6737b0e5efce913` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:57.739909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962ad4a-2202-4afa-9460-3d67770ade20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23b3bfaaf2974d51a6737b0e5efce913)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:57.741965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962ad4a-2202-4afa-9460-3d67770ade20 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:57.803067 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3bf7aac7-2148-4161-af97-6d37dbaa2b83 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:57.804417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3bf7aac7-2148-4161-af97-6d37dbaa2b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:57.872936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 852/1716] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:57.881863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7aac7-2148-4161-af97-6d37dbaa2b83 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:57.916177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7aac7-2148-4161-af97-6d37dbaa2b83 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:57.954732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba95e3-d828-4dca-800c-fb56d85c0131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:57.955788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba95e3-d828-4dca-800c-fb56d85c0131 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:57.956218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba95e3-d828-4dca-800c-fb56d85c0131 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:57.956639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba95e3-d828-4dca-800c-fb56d85c0131 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:57.957531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba95e3-d828-4dca-800c-fb56d85c0131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:59:57.958212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba95e3-d828-4dca-800c-fb56d85c0131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:57.960434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba95e3-d828-4dca-800c-fb56d85c0131 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:58.044569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 853/1717] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 17:59:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:59:58.045795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba95e3-d828-4dca-800c-fb56d85c0131 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:58.073668 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f962ad4a-2202-4afa-9460-3d67770ade20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23b3bfaaf2974d51a6737b0e5efce913', 'request_id': 'req-f962ad4a-2202-4afa-9460-3d67770ade20'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:58.122597 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba95e3-d828-4dca-800c-fb56d85c0131 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:58.162454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:58.207286 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f962ad4a-2202-4afa-9460-3d67770ade20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23b3bfaaf2974d51a6737b0e5efce913', 'request_id': 'req-f962ad4a-2202-4afa-9460-3d67770ade20'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:58.216811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f962ad4a-2202-4afa-9460-3d67770ade20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23b3bfaaf2974d51a6737b0e5efce913', 'request_id': 'req-f962ad4a-2202-4afa-9460-3d67770ade20'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:58.264347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f962ad4a-2202-4afa-9460-3d67770ade20 admin admin] The token cache is being invalidate because project 23b3bfaaf2974d51a6737b0e5efce913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 17:59:58.271466 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f962ad4a-2202-4afa-9460-3d67770ade20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:58.274436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f962ad4a-2202-4afa-9460-3d67770ade20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:58.282919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 865/1718] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 17:59:57 2022] DELETE /identity/v3/projects/23b3bfaaf2974d51a6737b0e5efce913 => generated 0 bytes in 870 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:58.288548 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b0401-01e8-42de-ab8e-582e825b74e9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:58.288894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b0401-01e8-42de-ab8e-582e825b74e9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:58.289274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b0401-01e8-42de-ab8e-582e825b74e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:58.331056 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-530b0401-01e8-42de-ab8e-582e825b74e9 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:58.333562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33399a4-d0bc-40c5-8f5c-9ce13ec8a5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:58.334674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33399a4-d0bc-40c5-8f5c-9ce13ec8a5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:58.335252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33399a4-d0bc-40c5-8f5c-9ce13ec8a5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:58.335778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33399a4-d0bc-40c5-8f5c-9ce13ec8a5b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:58.336773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33399a4-d0bc-40c5-8f5c-9ce13ec8a5b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:58.339653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33399a4-d0bc-40c5-8f5c-9ce13ec8a5b6 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:58.342589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-530b0401-01e8-42de-ab8e-582e825b74e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:58.368252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 866/1719] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:58.378300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e73b23-0bb1-4eb3-9eff-911b557b49cf None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:58.378687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e73b23-0bb1-4eb3-9eff-911b557b49cf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:58.379089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e73b23-0bb1-4eb3-9eff-911b557b49cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:58.416700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a1e73b23-0bb1-4eb3-9eff-911b557b49cf None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:58.423161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a1e73b23-0bb1-4eb3-9eff-911b557b49cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:58.447398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 867/1720] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 17:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:58.452550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e73b23-0bb1-4eb3-9eff-911b557b49cf None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:58.481825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e73b23-0bb1-4eb3-9eff-911b557b49cf None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:58.503175 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 854/1721] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:58 2022] POST /identity/v3/projects => generated 364 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:58.504558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33399a4-d0bc-40c5-8f5c-9ce13ec8a5b6 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:58.522676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c33399a4-d0bc-40c5-8f5c-9ce13ec8a5b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:58.540112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d407d-5799-44dc-8d64-da88516009fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:58.541207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d407d-5799-44dc-8d64-da88516009fb admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:58.543747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d407d-5799-44dc-8d64-da88516009fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:58.544153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d407d-5799-44dc-8d64-da88516009fb admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:58.551138 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d407d-5799-44dc-8d64-da88516009fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:59:58.552083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d407d-5799-44dc-8d64-da88516009fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:58.564108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d407d-5799-44dc-8d64-da88516009fb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:58.600369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 868/1722] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 17:59:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:59:58.604836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d407d-5799-44dc-8d64-da88516009fb admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:58.639271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d407d-5799-44dc-8d64-da88516009fb admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:58.719421 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:58.800156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33399a4-d0bc-40c5-8f5c-9ce13ec8a5b6 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:58.820997 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9b6ed-b53b-43a9-9b14-27d8a02a1883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:58.822137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9b6ed-b53b-43a9-9b14-27d8a02a1883 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:58.822624 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9b6ed-b53b-43a9-9b14-27d8a02a1883 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:58.823185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9b6ed-b53b-43a9-9b14-27d8a02a1883 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:58.824136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9b6ed-b53b-43a9-9b14-27d8a02a1883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:58.826422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9b6ed-b53b-43a9-9b14-27d8a02a1883 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:58.900203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4e3b8-dcdc-4a3b-9bc8-69b94bccd9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:58.901188 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4e3b8-dcdc-4a3b-9bc8-69b94bccd9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:58.909236 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4e3b8-dcdc-4a3b-9bc8-69b94bccd9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:58.909787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4e3b8-dcdc-4a3b-9bc8-69b94bccd9b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:58.910831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4e3b8-dcdc-4a3b-9bc8-69b94bccd9b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:58.915475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4e3b8-dcdc-4a3b-9bc8-69b94bccd9b2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:58.970870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 855/1723] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:58 2022] POST /identity/v3/users => generated 283 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:58.976023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9b6ed-b53b-43a9-9b14-27d8a02a1883 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:59.020050 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9b6ed-b53b-43a9-9b14-27d8a02a1883 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:59.055736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16454554-d744-449f-955b-49f751bc85d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:59.056927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16454554-d744-449f-955b-49f751bc85d7 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:59.057519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16454554-d744-449f-955b-49f751bc85d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:59.058184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16454554-d744-449f-955b-49f751bc85d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:59.059226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 869/1724] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 17:59:58 2022] POST /identity/v3/projects => generated 394 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:59.060179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4e3b8-dcdc-4a3b-9bc8-69b94bccd9b2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:59.065014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16454554-d744-449f-955b-49f751bc85d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:59.067272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16454554-d744-449f-955b-49f751bc85d7 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:59.109199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 856/1725] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:58 2022] GET /identity/v3/roles => generated 1427 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:59.113320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16454554-d744-449f-955b-49f751bc85d7 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:59.135998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16454554-d744-449f-955b-49f751bc85d7 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:59.315511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f394ca-85a4-4f4c-a7a2-b32d163ec02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:59.319206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f394ca-85a4-4f4c-a7a2-b32d163ec02f admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:59.319497 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f394ca-85a4-4f4c-a7a2-b32d163ec02f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:59.321384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f394ca-85a4-4f4c-a7a2-b32d163ec02f admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:59.335357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f394ca-85a4-4f4c-a7a2-b32d163ec02f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:59.340566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f394ca-85a4-4f4c-a7a2-b32d163ec02f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:59.358541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4e3b8-dcdc-4a3b-9bc8-69b94bccd9b2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:59.368686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 857/1726] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:59 2022] GET /identity/v3/roles => generated 1427 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:59:59.372936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f394ca-85a4-4f4c-a7a2-b32d163ec02f admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:59.373126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf48675-33f3-4fb0-844e-8abf37ee302e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:59.373835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf48675-33f3-4fb0-844e-8abf37ee302e admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:59.374117 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf48675-33f3-4fb0-844e-8abf37ee302e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:59.375381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf48675-33f3-4fb0-844e-8abf37ee302e admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:59.376063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf48675-33f3-4fb0-844e-8abf37ee302e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:59.378007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf48675-33f3-4fb0-844e-8abf37ee302e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:59.396100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f394ca-85a4-4f4c-a7a2-b32d163ec02f admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:59.410537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543a1f0-88a3-4cee-8cdf-f162dc4cdff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:59.412227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543a1f0-88a3-4cee-8cdf-f162dc4cdff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:59.413460 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543a1f0-88a3-4cee-8cdf-f162dc4cdff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:59.413759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543a1f0-88a3-4cee-8cdf-f162dc4cdff2 admin admin] PATH_INFO: `/v3/projects/2532e977a325486b923404317ed11c87/users/0b53053fd46c441d9a62c02e43a7fa50/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:59.464851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543a1f0-88a3-4cee-8cdf-f162dc4cdff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2532e977a325486b923404317ed11c87, user_id=0b53053fd46c441d9a62c02e43a7fa50, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:59.473456 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543a1f0-88a3-4cee-8cdf-f162dc4cdff2 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:59.548503 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 870/1727] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 17:59:59 2022] POST /identity/v3/users => generated 298 bytes in 489 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:59:59.550026 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf48675-33f3-4fb0-844e-8abf37ee302e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:59.556346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0bf48675-33f3-4fb0-844e-8abf37ee302e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:59.600925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf48675-33f3-4fb0-844e-8abf37ee302e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:59.612944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684987d-431a-469c-935c-2c438b96b6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:59.614021 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684987d-431a-469c-935c-2c438b96b6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:59.617289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684987d-431a-469c-935c-2c438b96b6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:59.617654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684987d-431a-469c-935c-2c438b96b6e5 admin admin] PATH_INFO: `/v3/projects/6251e828bb784b898ecf49686477f863` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:59.620065 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684987d-431a-469c-935c-2c438b96b6e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6251e828bb784b898ecf49686477f863)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:59.627218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684987d-431a-469c-935c-2c438b96b6e5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:59.653878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 858/1728] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 17:59:59 2022] PUT /identity/v3/projects/2532e977a325486b923404317ed11c87/users/0b53053fd46c441d9a62c02e43a7fa50/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:59:59.657787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543a1f0-88a3-4cee-8cdf-f162dc4cdff2 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:59.852352 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0684987d-431a-469c-935c-2c438b96b6e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6251e828bb784b898ecf49686477f863', 'request_id': 'req-0684987d-431a-469c-935c-2c438b96b6e5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:59.862216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543a1f0-88a3-4cee-8cdf-f162dc4cdff2 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:59.931506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0684987d-431a-469c-935c-2c438b96b6e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6251e828bb784b898ecf49686477f863', 'request_id': 'req-0684987d-431a-469c-935c-2c438b96b6e5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:59.945665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0684987d-431a-469c-935c-2c438b96b6e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6251e828bb784b898ecf49686477f863', 'request_id': 'req-0684987d-431a-469c-935c-2c438b96b6e5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 17:59:59.991971 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0684987d-431a-469c-935c-2c438b96b6e5 admin admin] The token cache is being invalidate because project 6251e828bb784b898ecf49686477f863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 17:59:59.992951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0684987d-431a-469c-935c-2c438b96b6e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:00.006717 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0684987d-431a-469c-935c-2c438b96b6e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:00.009625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 871/1729] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 17:59:59 2022] DELETE /identity/v3/projects/6251e828bb784b898ecf49686477f863 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:00.024037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ed276-f8ee-4585-9d91-18f87c833984 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:00.026790 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ed276-f8ee-4585-9d91-18f87c833984 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:00.028775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ed276-f8ee-4585-9d91-18f87c833984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:00.067412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95590d11-20fb-4c08-89f7-770a64e26224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:00.068185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95590d11-20fb-4c08-89f7-770a64e26224 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:00.068532 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95590d11-20fb-4c08-89f7-770a64e26224 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:00.072043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95590d11-20fb-4c08-89f7-770a64e26224 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:00.072043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95590d11-20fb-4c08-89f7-770a64e26224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:00.073771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95590d11-20fb-4c08-89f7-770a64e26224 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:00.110985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 859/1730] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 17:59:59 2022] GET /identity/v3/roles => generated 1427 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:00.110985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6fb96-917d-468f-bb87-7ab66e9a576a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:00.110985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6fb96-917d-468f-bb87-7ab66e9a576a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:00.110985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6fb96-917d-468f-bb87-7ab66e9a576a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:00.147671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-35f6fb96-917d-468f-bb87-7ab66e9a576a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:00.149454 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-267ed276-f8ee-4585-9d91-18f87c833984 None None] MFA Rules not processed for user `0b53053fd46c441d9a62c02e43a7fa50`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:00.150413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-35f6fb96-917d-468f-bb87-7ab66e9a576a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:00.153816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-267ed276-f8ee-4585-9d91-18f87c833984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:00.158503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:00.191274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 860/1731] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:00.216029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6fb96-917d-468f-bb87-7ab66e9a576a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:00.219883 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-35f6fb96-917d-468f-bb87-7ab66e9a576a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:00.275744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6fb96-917d-468f-bb87-7ab66e9a576a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:00.299586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5a4e5-c00d-44d9-b147-32e1592108ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:00.300017 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5a4e5-c00d-44d9-b147-32e1592108ea admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:00.301546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5a4e5-c00d-44d9-b147-32e1592108ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:00.301887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5a4e5-c00d-44d9-b147-32e1592108ea admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:00.302678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5a4e5-c00d-44d9-b147-32e1592108ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:00.307451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5a4e5-c00d-44d9-b147-32e1592108ea admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:00.335014 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 872/1732] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:00.336539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f344-ce64-4221-ad90-6093cd4ca5e7 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:00.336838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f344-ce64-4221-ad90-6093cd4ca5e7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:00.337116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f344-ce64-4221-ad90-6093cd4ca5e7 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:00.339579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 873/1733] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 18:00:00 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 18:00:00.342414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9375dc3-213f-4c79-8bd0-0f56178a2c96 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:00.348275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9375dc3-213f-4c79-8bd0-0f56178a2c96 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:00.348275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9375dc3-213f-4c79-8bd0-0f56178a2c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:00.349605 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 861/1734] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:00 2022] GET /identity/v3/roles => generated 1427 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:00.351190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5a4e5-c00d-44d9-b147-32e1592108ea admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:00.353282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-34d5a4e5-c00d-44d9-b147-32e1592108ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:00.384857 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5a4e5-c00d-44d9-b147-32e1592108ea admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:00.397687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c9375dc3-213f-4c79-8bd0-0f56178a2c96 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:00.401084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c9375dc3-213f-4c79-8bd0-0f56178a2c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:00.403000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d067344-ba1e-4a6b-9dda-f8aa51ce03a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:00.420592 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d067344-ba1e-4a6b-9dda-f8aa51ce03a6 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:00.426721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d067344-ba1e-4a6b-9dda-f8aa51ce03a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:00.426721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d067344-ba1e-4a6b-9dda-f8aa51ce03a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:00.426976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 874/1735] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:00.426976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9375dc3-213f-4c79-8bd0-0f56178a2c96 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:00.438306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d067344-ba1e-4a6b-9dda-f8aa51ce03a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:00.440987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:00.443408 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d067344-ba1e-4a6b-9dda-f8aa51ce03a6 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:00.460591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9375dc3-213f-4c79-8bd0-0f56178a2c96 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:00.482328 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d22e5-7c27-454d-8a82-81c044db99d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:00.484934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d22e5-7c27-454d-8a82-81c044db99d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:00.485471 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d22e5-7c27-454d-8a82-81c044db99d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:00.486512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d22e5-7c27-454d-8a82-81c044db99d4 admin admin] PATH_INFO: `/v3/projects/6693c9fed829462b95fb8db250d95ccc/users/da9769988c8a474099ad45f7e006a8ef/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:00.541597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d22e5-7c27-454d-8a82-81c044db99d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6693c9fed829462b95fb8db250d95ccc, user_id=da9769988c8a474099ad45f7e006a8ef, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:00.547378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d22e5-7c27-454d-8a82-81c044db99d4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:00.602929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 862/1736] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:00 2022] POST /identity/v3/projects => generated 366 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:00.609053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d067344-ba1e-4a6b-9dda-f8aa51ce03a6 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:00.665315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 875/1737] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:00 2022] PUT /identity/v3/projects/6693c9fed829462b95fb8db250d95ccc/users/da9769988c8a474099ad45f7e006a8ef/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:00.667081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372ca8a-d49e-4962-b741-ecfc38a2ff9e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:00.667372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372ca8a-d49e-4962-b741-ecfc38a2ff9e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:00.667648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372ca8a-d49e-4962-b741-ecfc38a2ff9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:00.714325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b372ca8a-d49e-4962-b741-ecfc38a2ff9e None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:00.716063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b372ca8a-d49e-4962-b741-ecfc38a2ff9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:00.853733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d067344-ba1e-4a6b-9dda-f8aa51ce03a6 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:00.923888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 876/1738] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 18:00:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:00.925131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372ca8a-d49e-4962-b741-ecfc38a2ff9e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:00.943114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:01.118247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376c095-36e7-4ac4-97ce-9e139cbaddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:01.129318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376c095-36e7-4ac4-97ce-9e139cbaddfa admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:01.129896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376c095-36e7-4ac4-97ce-9e139cbaddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:01.130352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376c095-36e7-4ac4-97ce-9e139cbaddfa admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:01.134534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376c095-36e7-4ac4-97ce-9e139cbaddfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:01.135545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376c095-36e7-4ac4-97ce-9e139cbaddfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:01.137676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376c095-36e7-4ac4-97ce-9e139cbaddfa admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:01.142268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372ca8a-d49e-4962-b741-ecfc38a2ff9e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:01.160092 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3a23c-3ea1-4cd8-9088-03fe879e85e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:01.160842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3a23c-3ea1-4cd8-9088-03fe879e85e6 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:01.161124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3a23c-3ea1-4cd8-9088-03fe879e85e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:01.161399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3a23c-3ea1-4cd8-9088-03fe879e85e6 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:01.162335 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3a23c-3ea1-4cd8-9088-03fe879e85e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:01.164332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3a23c-3ea1-4cd8-9088-03fe879e85e6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:01.188890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 863/1739] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 583 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:01.190615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b46c7f-1cfc-42cf-b311-8250edd499d6 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:01.194931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b46c7f-1cfc-42cf-b311-8250edd499d6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:01.195781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b46c7f-1cfc-42cf-b311-8250edd499d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:01.282981 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a2b46c7f-1cfc-42cf-b311-8250edd499d6 None None] MFA Rules not processed for user `da9769988c8a474099ad45f7e006a8ef`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:01.286434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a2b46c7f-1cfc-42cf-b311-8250edd499d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:01.322801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 877/1740] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:00 2022] POST /identity/v3/users => generated 290 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:01.323788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3a23c-3ea1-4cd8-9088-03fe879e85e6 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:01.326540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-deb3a23c-3ea1-4cd8-9088-03fe879e85e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:01.363894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3a23c-3ea1-4cd8-9088-03fe879e85e6 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:01.377961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035b000-711e-4255-bcbd-58a3173e2483 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:01.378851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035b000-711e-4255-bcbd-58a3173e2483 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:01.379066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035b000-711e-4255-bcbd-58a3173e2483 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:01.379522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035b000-711e-4255-bcbd-58a3173e2483 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:01.422254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035b000-711e-4255-bcbd-58a3173e2483 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:01.428076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035b000-711e-4255-bcbd-58a3173e2483 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:01.468955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 878/1741] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:01.469855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035b000-711e-4255-bcbd-58a3173e2483 service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:01.480022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 864/1742] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:01.484405 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b46c7f-1cfc-42cf-b311-8250edd499d6 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:01.493179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:01.512560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b46c7f-1cfc-42cf-b311-8250edd499d6 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:01.517303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035b000-711e-4255-bcbd-58a3173e2483 service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:01.532441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f719a0-6274-4023-80ce-5b104882e169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:01.533265 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f719a0-6274-4023-80ce-5b104882e169 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:01.533667 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f719a0-6274-4023-80ce-5b104882e169 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:01.534879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f719a0-6274-4023-80ce-5b104882e169 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:01.535740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f719a0-6274-4023-80ce-5b104882e169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:01.537627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60814142-fbfe-4a1f-a861-d2f2f853a2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:01.540131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60814142-fbfe-4a1f-a861-d2f2f853a2da admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:01.540426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60814142-fbfe-4a1f-a861-d2f2f853a2da admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:01.540693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60814142-fbfe-4a1f-a861-d2f2f853a2da admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:01.541115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f719a0-6274-4023-80ce-5b104882e169 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:01.541407 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60814142-fbfe-4a1f-a861-d2f2f853a2da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:01.549858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60814142-fbfe-4a1f-a861-d2f2f853a2da admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:01.571319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 865/1743] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:01 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:01.573802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f719a0-6274-4023-80ce-5b104882e169 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:01.599561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f719a0-6274-4023-80ce-5b104882e169 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:01.605654 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7f7e0-e6e5-4149-9ec5-24044f7729bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:01.606780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7f7e0-e6e5-4149-9ec5-24044f7729bd admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:01.607461 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7f7e0-e6e5-4149-9ec5-24044f7729bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:01.608047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7f7e0-e6e5-4149-9ec5-24044f7729bd admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:01.610547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:01.611091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7f7e0-e6e5-4149-9ec5-24044f7729bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:01.614682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7f7e0-e6e5-4149-9ec5-24044f7729bd admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:01.676039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 879/1744] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:01 2022] POST /identity/v3/projects => generated 380 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:01.677666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60814142-fbfe-4a1f-a861-d2f2f853a2da admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:01.701308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 866/1745] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:01 2022] POST /identity/v3/projects => generated 394 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:01.702286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7f7e0-e6e5-4149-9ec5-24044f7729bd admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:01.948373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60814142-fbfe-4a1f-a861-d2f2f853a2da admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:01.955644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7f7e0-e6e5-4149-9ec5-24044f7729bd admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:01.960406 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487491f2-d671-4940-b27b-f485dccfd14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:01.975554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487491f2-d671-4940-b27b-f485dccfd14b admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:01.976490 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487491f2-d671-4940-b27b-f485dccfd14b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:01.977370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487491f2-d671-4940-b27b-f485dccfd14b admin admin] PATH_INFO: `/v3/projects/de1996aa222a4670bb5d42d4b8b3aab6/users/fd5ffe96753843ee9a174aca22b4c416/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:01.982261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46ad68-de58-4033-81ad-9211b4a08b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:01.983546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46ad68-de58-4033-81ad-9211b4a08b20 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:01.984830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46ad68-de58-4033-81ad-9211b4a08b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:01.986882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46ad68-de58-4033-81ad-9211b4a08b20 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:01.988816 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46ad68-de58-4033-81ad-9211b4a08b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:02.003321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46ad68-de58-4033-81ad-9211b4a08b20 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:02.036416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487491f2-d671-4940-b27b-f485dccfd14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1996aa222a4670bb5d42d4b8b3aab6, user_id=fd5ffe96753843ee9a174aca22b4c416, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:02.042353 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487491f2-d671-4940-b27b-f485dccfd14b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:02.119494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 867/1746] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:01 2022] POST /identity/v3/users => generated 291 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:02.120882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46ad68-de58-4033-81ad-9211b4a08b20 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:02.144470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 880/1747] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:01 2022] PUT /identity/v3/projects/de1996aa222a4670bb5d42d4b8b3aab6/users/fd5ffe96753843ee9a174aca22b4c416/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:02.145840 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487491f2-d671-4940-b27b-f485dccfd14b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:02.190950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46ad68-de58-4033-81ad-9211b4a08b20 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:02.431521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487491f2-d671-4940-b27b-f485dccfd14b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:02.439792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b85a00-44f9-44b6-9829-3b565f47ae92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:02.442788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b85a00-44f9-44b6-9829-3b565f47ae92 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:02.444004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b85a00-44f9-44b6-9829-3b565f47ae92 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:02.444325 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b85a00-44f9-44b6-9829-3b565f47ae92 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:02.444631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c3890-9069-4bbc-903d-5401542a4092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:02.447163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b85a00-44f9-44b6-9829-3b565f47ae92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:02.447716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c3890-9069-4bbc-903d-5401542a4092 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:02.449295 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c3890-9069-4bbc-903d-5401542a4092 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:02.449537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c3890-9069-4bbc-903d-5401542a4092 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:02.451535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b85a00-44f9-44b6-9829-3b565f47ae92 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:02.452197 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c3890-9069-4bbc-903d-5401542a4092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:02.455887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c3890-9069-4bbc-903d-5401542a4092 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:02.480712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 881/1748] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:02 2022] GET /identity/v3/roles => generated 1427 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:02.483886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c3890-9069-4bbc-903d-5401542a4092 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:02.506645 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c3890-9069-4bbc-903d-5401542a4092 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:02.522340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b675573-8e45-41c7-95df-aaf853a988ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:02.523560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b675573-8e45-41c7-95df-aaf853a988ee admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:02.526293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b675573-8e45-41c7-95df-aaf853a988ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:02.527227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b675573-8e45-41c7-95df-aaf853a988ee admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:02.528125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b675573-8e45-41c7-95df-aaf853a988ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:02.530318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b675573-8e45-41c7-95df-aaf853a988ee admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:02.563108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 882/1749] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:02 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:02.564238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b675573-8e45-41c7-95df-aaf853a988ee admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:02.570725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 868/1750] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:02 2022] POST /identity/v3/users => generated 304 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:02.573650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b85a00-44f9-44b6-9829-3b565f47ae92 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:02.583342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b675573-8e45-41c7-95df-aaf853a988ee admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:02.597377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48540fb0-9018-4de3-9d1b-049d2d835e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:02.598396 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48540fb0-9018-4de3-9d1b-049d2d835e0a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:02.598540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48540fb0-9018-4de3-9d1b-049d2d835e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:02.598807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48540fb0-9018-4de3-9d1b-049d2d835e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:02.599741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48540fb0-9018-4de3-9d1b-049d2d835e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:02.601519 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48540fb0-9018-4de3-9d1b-049d2d835e0a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:02.619645 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b85a00-44f9-44b6-9829-3b565f47ae92 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:02.630257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 883/1751] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:02 2022] GET /identity/v3/roles => generated 1427 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:02.630257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48540fb0-9018-4de3-9d1b-049d2d835e0a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:02.634682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2af11-f3ff-4d02-802a-c86654af5656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:02.637657 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2af11-f3ff-4d02-802a-c86654af5656 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:02.639321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2af11-f3ff-4d02-802a-c86654af5656 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:02.639321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2af11-f3ff-4d02-802a-c86654af5656 admin admin] PATH_INFO: `/v3/projects/de1996aa222a4670bb5d42d4b8b3aab6/users/fd5ffe96753843ee9a174aca22b4c416/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:02.643723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48540fb0-9018-4de3-9d1b-049d2d835e0a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:02.643825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2af11-f3ff-4d02-802a-c86654af5656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1996aa222a4670bb5d42d4b8b3aab6, user_id=fd5ffe96753843ee9a174aca22b4c416, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:02.652634 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb723fd3-9255-42f0-b73d-e4d3d2a0b8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:02.653712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb723fd3-9255-42f0-b73d-e4d3d2a0b8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:02.654134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb723fd3-9255-42f0-b73d-e4d3d2a0b8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:02.654600 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb723fd3-9255-42f0-b73d-e4d3d2a0b8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:02.655572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb723fd3-9255-42f0-b73d-e4d3d2a0b8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:02.661274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2af11-f3ff-4d02-802a-c86654af5656 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:02.671330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb723fd3-9255-42f0-b73d-e4d3d2a0b8a1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:02.681101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:02.700795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 884/1752] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:02 2022] GET /identity/v3/roles => generated 1427 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:02.713788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 869/1753] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:02 2022] PUT /identity/v3/projects/de1996aa222a4670bb5d42d4b8b3aab6/users/fd5ffe96753843ee9a174aca22b4c416/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:02.730020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2af11-f3ff-4d02-802a-c86654af5656 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:02.740340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb723fd3-9255-42f0-b73d-e4d3d2a0b8a1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:02.763392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:02.805175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb723fd3-9255-42f0-b73d-e4d3d2a0b8a1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:02.873799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2af11-f3ff-4d02-802a-c86654af5656 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:02.880589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad258452-578f-4cae-b5e9-de0447bda17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:02.881531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad258452-578f-4cae-b5e9-de0447bda17e admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:02.881796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad258452-578f-4cae-b5e9-de0447bda17e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:02.882057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad258452-578f-4cae-b5e9-de0447bda17e admin admin] PATH_INFO: `/v3/projects/2ce08c459e734bdc81d35996f3cd8274/users/72b560db4df84b47b033f89cad737b0f/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:02.900059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad258452-578f-4cae-b5e9-de0447bda17e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce08c459e734bdc81d35996f3cd8274, user_id=72b560db4df84b47b033f89cad737b0f, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:02.906806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad258452-578f-4cae-b5e9-de0447bda17e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:03.041413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 870/1754] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:02 2022] PUT /identity/v3/projects/2ce08c459e734bdc81d35996f3cd8274/users/72b560db4df84b47b033f89cad737b0f/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:03.048083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef320a59-9889-4fd8-8678-52971062666d None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:03.049302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef320a59-9889-4fd8-8678-52971062666d None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:03.049798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef320a59-9889-4fd8-8678-52971062666d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:03.051262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d89de5-bc8f-4510-b30e-1ca5a8ca875e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:03.052443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d89de5-bc8f-4510-b30e-1ca5a8ca875e admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:03.052795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d89de5-bc8f-4510-b30e-1ca5a8ca875e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:03.053088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d89de5-bc8f-4510-b30e-1ca5a8ca875e admin admin] PATH_INFO: `/v3/projects/8a83868422324650bcdc52ad61be2058/users/762e848fc3aa41a6a94e4bec2e908866/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:03.094422 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d89de5-bc8f-4510-b30e-1ca5a8ca875e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a83868422324650bcdc52ad61be2058, user_id=762e848fc3aa41a6a94e4bec2e908866, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:03.097615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d89de5-bc8f-4510-b30e-1ca5a8ca875e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:03.124113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-ef320a59-9889-4fd8-8678-52971062666d None None] MFA Rules not processed for user `fd5ffe96753843ee9a174aca22b4c416`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:03.132675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ef320a59-9889-4fd8-8678-52971062666d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:03.215311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 885/1755] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:02 2022] PUT /identity/v3/projects/8a83868422324650bcdc52ad61be2058/users/762e848fc3aa41a6a94e4bec2e908866/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:03.217415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d89de5-bc8f-4510-b30e-1ca5a8ca875e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:03.382928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 871/1756] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:03.389194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911db9e4-bde9-4f48-bcaa-092826644e98 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:03.389488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911db9e4-bde9-4f48-bcaa-092826644e98 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:03.389815 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911db9e4-bde9-4f48-bcaa-092826644e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:03.464357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d89de5-bc8f-4510-b30e-1ca5a8ca875e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:03.469759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-911db9e4-bde9-4f48-bcaa-092826644e98 None None] MFA Rules not processed for user `762e848fc3aa41a6a94e4bec2e908866`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:03.470301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674265bf-2f8b-4b3b-b3a5-bc68145f6596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:03.471470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674265bf-2f8b-4b3b-b3a5-bc68145f6596 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:03.472001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674265bf-2f8b-4b3b-b3a5-bc68145f6596 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:03.472149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-911db9e4-bde9-4f48-bcaa-092826644e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:03.483283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674265bf-2f8b-4b3b-b3a5-bc68145f6596 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:03.483283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674265bf-2f8b-4b3b-b3a5-bc68145f6596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:03.483283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674265bf-2f8b-4b3b-b3a5-bc68145f6596 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:03.492479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 886/1757] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:03 2022] GET /identity/v3/roles => generated 1427 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:03.494243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0b1e8-fbb9-44a5-93e2-857b4c319e10 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:03.494557 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0b1e8-fbb9-44a5-93e2-857b4c319e10 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:03.494904 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0b1e8-fbb9-44a5-93e2-857b4c319e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:03.517673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e5a0b1e8-fbb9-44a5-93e2-857b4c319e10 None None] MFA Rules not processed for user `fe1c86c411834f1c8d2775a5b2c5f1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:03.520624 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e5a0b1e8-fbb9-44a5-93e2-857b4c319e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:03.653707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 887/1758] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:03.654717 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0b1e8-fbb9-44a5-93e2-857b4c319e10 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:03.657396 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e5a0b1e8-fbb9-44a5-93e2-857b4c319e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:03.729843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 872/1759] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:03.739190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911db9e4-bde9-4f48-bcaa-092826644e98 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:03.761941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:03.775490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911db9e4-bde9-4f48-bcaa-092826644e98 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:03.788845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb9755-ad3d-42bb-b3ee-e9c1983f67d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:03.804293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0b1e8-fbb9-44a5-93e2-857b4c319e10 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:03.818152 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:03.820951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d685ac-3d76-4c24-9e9d-5127ab3694ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:03.821769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb9755-ad3d-42bb-b3ee-e9c1983f67d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:03.824076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d685ac-3d76-4c24-9e9d-5127ab3694ad service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:03.830956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb9755-ad3d-42bb-b3ee-e9c1983f67d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:03.831581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d685ac-3d76-4c24-9e9d-5127ab3694ad service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:03.831786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb9755-ad3d-42bb-b3ee-e9c1983f67d5 admin admin] PATH_INFO: `/v3/projects/2ce08c459e734bdc81d35996f3cd8274/users/72b560db4df84b47b033f89cad737b0f/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:03.831870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d685ac-3d76-4c24-9e9d-5127ab3694ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:03.845225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb9755-ad3d-42bb-b3ee-e9c1983f67d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce08c459e734bdc81d35996f3cd8274, user_id=72b560db4df84b47b033f89cad737b0f, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:03.856465 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb9755-ad3d-42bb-b3ee-e9c1983f67d5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:03.910980 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 873/1760] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:03 2022] PUT /identity/v3/projects/2ce08c459e734bdc81d35996f3cd8274/users/72b560db4df84b47b033f89cad737b0f/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:03.913269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb9755-ad3d-42bb-b3ee-e9c1983f67d5 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:04.032179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d685ac-3d76-4c24-9e9d-5127ab3694ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:04.035197 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d685ac-3d76-4c24-9e9d-5127ab3694ad service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:04.070248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 888/1761] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:04.076298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b981316-237d-448b-bf7f-ecb3bcd4b383 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:04.078202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b981316-237d-448b-bf7f-ecb3bcd4b383 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:04.078539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b981316-237d-448b-bf7f-ecb3bcd4b383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:04.092317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb9755-ad3d-42bb-b3ee-e9c1983f67d5 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:04.098214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fec67-1616-4d05-a142-b874118b1445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:04.098993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fec67-1616-4d05-a142-b874118b1445 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:04.099282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fec67-1616-4d05-a142-b874118b1445 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:04.099591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fec67-1616-4d05-a142-b874118b1445 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:04.100295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fec67-1616-4d05-a142-b874118b1445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:04.102396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fec67-1616-4d05-a142-b874118b1445 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:04.197856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8b981316-237d-448b-bf7f-ecb3bcd4b383 None None] MFA Rules not processed for user `72b560db4df84b47b033f89cad737b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:04.203681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8b981316-237d-448b-bf7f-ecb3bcd4b383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:04.203851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 874/1762] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:03 2022] POST /identity/v3/projects => generated 382 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:04.205074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fec67-1616-4d05-a142-b874118b1445 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:04.407880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 889/1763] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:04 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:04.420820 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b981316-237d-448b-bf7f-ecb3bcd4b383 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:04.516474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fec67-1616-4d05-a142-b874118b1445 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:04.535371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:04.542471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72fb63-fb94-4969-94d8-d47df183928a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:04.545362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72fb63-fb94-4969-94d8-d47df183928a admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:04.546402 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72fb63-fb94-4969-94d8-d47df183928a admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:04.546774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72fb63-fb94-4969-94d8-d47df183928a admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:04.552233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72fb63-fb94-4969-94d8-d47df183928a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:04.560455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72fb63-fb94-4969-94d8-d47df183928a admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:04.609234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b981316-237d-448b-bf7f-ecb3bcd4b383 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:04.621243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8c2bb-8b2c-49e4-b64e-b55f758fb4d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:04.622648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8c2bb-8b2c-49e4-b64e-b55f758fb4d6 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:04.636559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8c2bb-8b2c-49e4-b64e-b55f758fb4d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:04.636992 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8c2bb-8b2c-49e4-b64e-b55f758fb4d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:04.685408 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 875/1764] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:04 2022] POST /identity/v3/users => generated 298 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:04.688993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8c2bb-8b2c-49e4-b64e-b55f758fb4d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:04.697067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8c2bb-8b2c-49e4-b64e-b55f758fb4d6 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:04.699888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72fb63-fb94-4969-94d8-d47df183928a admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:04.717937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72fb63-fb94-4969-94d8-d47df183928a admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:04.744649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 890/1765] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:04 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:04.765463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb6ea4-5812-4f47-b571-1c5313cd905f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:04.769752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb6ea4-5812-4f47-b571-1c5313cd905f admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:04.771212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb6ea4-5812-4f47-b571-1c5313cd905f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:04.772763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb6ea4-5812-4f47-b571-1c5313cd905f admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:04.775912 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb6ea4-5812-4f47-b571-1c5313cd905f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:04.778154 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb6ea4-5812-4f47-b571-1c5313cd905f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:04.831622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 876/1766] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:04 2022] GET /identity/v3/roles => generated 1427 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:04.832974 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8c2bb-8b2c-49e4-b64e-b55f758fb4d6 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:04.854299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8c2bb-8b2c-49e4-b64e-b55f758fb4d6 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:04.873337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67af17e-ba7a-4240-bd14-ea2a325a3981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:04.874706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67af17e-ba7a-4240-bd14-ea2a325a3981 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:04.875041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67af17e-ba7a-4240-bd14-ea2a325a3981 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:04.875391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67af17e-ba7a-4240-bd14-ea2a325a3981 admin admin] PATH_INFO: `/v3/projects/8616007633fb41bba2596844fa7e6e1f/users/4342f35dd86a4c2d861049482defd9ff/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:04.901872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67af17e-ba7a-4240-bd14-ea2a325a3981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8616007633fb41bba2596844fa7e6e1f, user_id=4342f35dd86a4c2d861049482defd9ff, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:04.904728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67af17e-ba7a-4240-bd14-ea2a325a3981 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:04.928715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb6ea4-5812-4f47-b571-1c5313cd905f admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:04.945995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb6ea4-5812-4f47-b571-1c5313cd905f admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:04.982655 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 891/1767] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:04 2022] PUT /identity/v3/projects/8616007633fb41bba2596844fa7e6e1f/users/4342f35dd86a4c2d861049482defd9ff/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:04.987395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1a490-b170-4534-931e-1a88c5d7c4f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:04.989168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1a490-b170-4534-931e-1a88c5d7c4f0 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:04.989914 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1a490-b170-4534-931e-1a88c5d7c4f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:04.990202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1a490-b170-4534-931e-1a88c5d7c4f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:04.992573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67af17e-ba7a-4240-bd14-ea2a325a3981 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:05.114549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:05.140882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:05.179160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1a490-b170-4534-931e-1a88c5d7c4f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:05.180503 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67af17e-ba7a-4240-bd14-ea2a325a3981 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:05.180716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1a490-b170-4534-931e-1a88c5d7c4f0 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:05.192853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96980e24-7a59-44ef-a5ee-b13eb8c0e32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:05.194491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96980e24-7a59-44ef-a5ee-b13eb8c0e32f admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:05.194778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96980e24-7a59-44ef-a5ee-b13eb8c0e32f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:05.195088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96980e24-7a59-44ef-a5ee-b13eb8c0e32f admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:05.195840 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96980e24-7a59-44ef-a5ee-b13eb8c0e32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:05.197477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96980e24-7a59-44ef-a5ee-b13eb8c0e32f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:05.219867 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 877/1768] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:04 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:05.230153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1a490-b170-4534-931e-1a88c5d7c4f0 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:05.231311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 892/1769] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:04 2022] GET /identity/v3/roles => generated 1427 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:05.260933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1a490-b170-4534-931e-1a88c5d7c4f0 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:05.269580 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625744be-eae1-4d4f-82d1-1a66f969c529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:05.278124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625744be-eae1-4d4f-82d1-1a66f969c529 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:05.278576 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625744be-eae1-4d4f-82d1-1a66f969c529 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:05.281023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625744be-eae1-4d4f-82d1-1a66f969c529 admin admin] PATH_INFO: `/v3/projects/8616007633fb41bba2596844fa7e6e1f/users/4342f35dd86a4c2d861049482defd9ff/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:05.286073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625744be-eae1-4d4f-82d1-1a66f969c529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8616007633fb41bba2596844fa7e6e1f, user_id=4342f35dd86a4c2d861049482defd9ff, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:05.292074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625744be-eae1-4d4f-82d1-1a66f969c529 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:05.316332 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 878/1770] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:05 2022] PUT /identity/v3/projects/8616007633fb41bba2596844fa7e6e1f/users/4342f35dd86a4c2d861049482defd9ff/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:05.330765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7dea4-fcc0-455b-a0e2-c8aef14dfb60 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:05.331075 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7dea4-fcc0-455b-a0e2-c8aef14dfb60 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:05.332010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7dea4-fcc0-455b-a0e2-c8aef14dfb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:05.422649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3cf7dea4-fcc0-455b-a0e2-c8aef14dfb60 None None] MFA Rules not processed for user `4342f35dd86a4c2d861049482defd9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:05.423925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3cf7dea4-fcc0-455b-a0e2-c8aef14dfb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:05.595154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 893/1771] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:05.625245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625744be-eae1-4d4f-82d1-1a66f969c529 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:05.627448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-625744be-eae1-4d4f-82d1-1a66f969c529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:05.808640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625744be-eae1-4d4f-82d1-1a66f969c529 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:05.814363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf697f-034a-4364-aff9-8d0afcaca79e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:05.815094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf697f-034a-4364-aff9-8d0afcaca79e service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:05.815376 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf697f-034a-4364-aff9-8d0afcaca79e service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:05.815651 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf697f-034a-4364-aff9-8d0afcaca79e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:05.839872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf26fc-c7b4-438b-b8c4-935e6cf404e3 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:05.840620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf26fc-c7b4-438b-b8c4-935e6cf404e3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:05.840620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf26fc-c7b4-438b-b8c4-935e6cf404e3 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:05.842275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 894/1772] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 18:00:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 18:00:05.843041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf697f-034a-4364-aff9-8d0afcaca79e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:05.845556 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf697f-034a-4364-aff9-8d0afcaca79e service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:05.869152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 879/1773] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:05.891011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2442e-6996-4984-b967-dc3b57764838 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:05.891707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2442e-6996-4984-b967-dc3b57764838 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:05.892165 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2442e-6996-4984-b967-dc3b57764838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:05.951973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e9a2442e-6996-4984-b967-dc3b57764838 None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:05.953732 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e9a2442e-6996-4984-b967-dc3b57764838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:06.186164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 895/1774] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 18:00:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:06.454286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf697f-034a-4364-aff9-8d0afcaca79e service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:06.474660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf697f-034a-4364-aff9-8d0afcaca79e service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:06.487786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468235e7-de0b-4961-b5b2-15fe6cebc5c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:06.488892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468235e7-de0b-4961-b5b2-15fe6cebc5c0 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:06.489254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468235e7-de0b-4961-b5b2-15fe6cebc5c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:06.489613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468235e7-de0b-4961-b5b2-15fe6cebc5c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:06.659390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468235e7-de0b-4961-b5b2-15fe6cebc5c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:06.661267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468235e7-de0b-4961-b5b2-15fe6cebc5c0 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:06.679539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 880/1775] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:06.970374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2442e-6996-4984-b967-dc3b57764838 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:06.990199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2442e-6996-4984-b967-dc3b57764838 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:07.001714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97a66f-d148-480e-8869-8667771c2e6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:07.003046 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97a66f-d148-480e-8869-8667771c2e6d service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:07.003716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97a66f-d148-480e-8869-8667771c2e6d service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:07.004300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97a66f-d148-480e-8869-8667771c2e6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:07.016834 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97a66f-d148-480e-8869-8667771c2e6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:07.019002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97a66f-d148-480e-8869-8667771c2e6d service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:07.040273 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 896/1776] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:07.388972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468235e7-de0b-4961-b5b2-15fe6cebc5c0 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:07.606895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468235e7-de0b-4961-b5b2-15fe6cebc5c0 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:07.616956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147c496-d2b4-4ee6-9734-3faf06a5e19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:07.618199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147c496-d2b4-4ee6-9734-3faf06a5e19d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:07.618647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147c496-d2b4-4ee6-9734-3faf06a5e19d admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:07.619141 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147c496-d2b4-4ee6-9734-3faf06a5e19d admin admin] PATH_INFO: `/v3/users/0b53053fd46c441d9a62c02e43a7fa50` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:07.620814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147c496-d2b4-4ee6-9734-3faf06a5e19d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b53053fd46c441d9a62c02e43a7fa50)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:07.623083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147c496-d2b4-4ee6-9734-3faf06a5e19d admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:07.660124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:07.842586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0147c496-d2b4-4ee6-9734-3faf06a5e19d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b53053fd46c441d9a62c02e43a7fa50'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:07.849891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0147c496-d2b4-4ee6-9734-3faf06a5e19d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b53053fd46c441d9a62c02e43a7fa50'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:07.850568 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0147c496-d2b4-4ee6-9734-3faf06a5e19d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b53053fd46c441d9a62c02e43a7fa50'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:07.862196 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0147c496-d2b4-4ee6-9734-3faf06a5e19d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b53053fd46c441d9a62c02e43a7fa50'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:07.951669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0147c496-d2b4-4ee6-9734-3faf06a5e19d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b53053fd46c441d9a62c02e43a7fa50'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:07.962678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 881/1777] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:07 2022] DELETE /identity/v3/users/0b53053fd46c441d9a62c02e43a7fa50 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:07.966317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421c902-1f0a-4d7b-9e2a-b0f4dcb3945c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:07.966648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421c902-1f0a-4d7b-9e2a-b0f4dcb3945c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:07.967116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421c902-1f0a-4d7b-9e2a-b0f4dcb3945c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:07.992774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2421c902-1f0a-4d7b-9e2a-b0f4dcb3945c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:07.996095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2421c902-1f0a-4d7b-9e2a-b0f4dcb3945c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:08.051860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:08.101214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b4aa-fb7f-453b-8c53-96b642f4e1e9 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:08.101493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b4aa-fb7f-453b-8c53-96b642f4e1e9 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:08.102533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b4aa-fb7f-453b-8c53-96b642f4e1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:08.131859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3d53b4aa-fb7f-453b-8c53-96b642f4e1e9 None None] MFA Rules not processed for user `4ffcb99923b346938eb72c0b2786e815`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:08.137398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3d53b4aa-fb7f-453b-8c53-96b642f4e1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:08.141323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 897/1778] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:08.152600 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421c902-1f0a-4d7b-9e2a-b0f4dcb3945c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:08.154207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2421c902-1f0a-4d7b-9e2a-b0f4dcb3945c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:08.220500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 882/1779] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:08.277255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421c902-1f0a-4d7b-9e2a-b0f4dcb3945c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:08.282400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665839c-8125-4040-b2e4-fae22cab6bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:08.283269 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665839c-8125-4040-b2e4-fae22cab6bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:08.283633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665839c-8125-4040-b2e4-fae22cab6bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:08.283987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665839c-8125-4040-b2e4-fae22cab6bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:08.295029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665839c-8125-4040-b2e4-fae22cab6bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:08.296623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665839c-8125-4040-b2e4-fae22cab6bf2 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:08.307775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 898/1780] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:09.410609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b4aa-fb7f-453b-8c53-96b642f4e1e9 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:09.413447 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3d53b4aa-fb7f-453b-8c53-96b642f4e1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:09.434716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b4aa-fb7f-453b-8c53-96b642f4e1e9 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:09.440075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:09.441480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:09.441948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:09.442408 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] PATH_INFO: `/v3/projects/2532e977a325486b923404317ed11c87` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:09.444047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2532e977a325486b923404317ed11c87)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:09.446403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:09.591081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2532e977a325486b923404317ed11c87', 'request_id': 'req-1f2478ad-349c-4120-9deb-d3706997f0f8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:09.651752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2532e977a325486b923404317ed11c87', 'request_id': 'req-1f2478ad-349c-4120-9deb-d3706997f0f8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:09.665109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2532e977a325486b923404317ed11c87', 'request_id': 'req-1f2478ad-349c-4120-9deb-d3706997f0f8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:09.677563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] The token cache is being invalidate because project 2532e977a325486b923404317ed11c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:09.677861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:09.679358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:09.682361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 883/1781] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:09 2022] DELETE /identity/v3/projects/2532e977a325486b923404317ed11c87 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:09.691979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665839c-8125-4040-b2e4-fae22cab6bf2 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:09.703939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665839c-8125-4040-b2e4-fae22cab6bf2 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:09.780765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcca7c-afe9-490b-bbd5-e3004135b305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:09.782366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcca7c-afe9-490b-bbd5-e3004135b305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:09.782647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcca7c-afe9-490b-bbd5-e3004135b305 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:09.782984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcca7c-afe9-490b-bbd5-e3004135b305 admin admin] PATH_INFO: `/v3/users/fd5ffe96753843ee9a174aca22b4c416` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:09.784218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcca7c-afe9-490b-bbd5-e3004135b305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5ffe96753843ee9a174aca22b4c416)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:09.786171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcca7c-afe9-490b-bbd5-e3004135b305 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:09.981644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-92bcca7c-afe9-490b-bbd5-e3004135b305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5ffe96753843ee9a174aca22b4c416'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:09.991146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-92bcca7c-afe9-490b-bbd5-e3004135b305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5ffe96753843ee9a174aca22b4c416'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:09.991851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-92bcca7c-afe9-490b-bbd5-e3004135b305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5ffe96753843ee9a174aca22b4c416'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:10.011778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-92bcca7c-afe9-490b-bbd5-e3004135b305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5ffe96753843ee9a174aca22b4c416'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:10.069973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-92bcca7c-afe9-490b-bbd5-e3004135b305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5ffe96753843ee9a174aca22b4c416'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:10.069973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:10.073628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 899/1782] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:09 2022] DELETE /identity/v3/users/fd5ffe96753843ee9a174aca22b4c416 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:10.503519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:10.505079 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:10.520440 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2478ad-349c-4120-9deb-d3706997f0f8 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:10.610994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c0652-5724-498b-b8e9-287f351be9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:10.613469 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c0652-5724-498b-b8e9-287f351be9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:10.613469 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c0652-5724-498b-b8e9-287f351be9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:10.613469 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c0652-5724-498b-b8e9-287f351be9ef admin admin] PATH_INFO: `/v3/projects/de1996aa222a4670bb5d42d4b8b3aab6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:10.615309 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c0652-5724-498b-b8e9-287f351be9ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1996aa222a4670bb5d42d4b8b3aab6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:10.617553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c0652-5724-498b-b8e9-287f351be9ef admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:10.740405 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-162c0652-5724-498b-b8e9-287f351be9ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1996aa222a4670bb5d42d4b8b3aab6', 'request_id': 'req-162c0652-5724-498b-b8e9-287f351be9ef'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:10.860150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-162c0652-5724-498b-b8e9-287f351be9ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1996aa222a4670bb5d42d4b8b3aab6', 'request_id': 'req-162c0652-5724-498b-b8e9-287f351be9ef'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:10.867601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-162c0652-5724-498b-b8e9-287f351be9ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1996aa222a4670bb5d42d4b8b3aab6', 'request_id': 'req-162c0652-5724-498b-b8e9-287f351be9ef'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:10.892147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-162c0652-5724-498b-b8e9-287f351be9ef admin admin] The token cache is being invalidate because project de1996aa222a4670bb5d42d4b8b3aab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:10.892672 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-162c0652-5724-498b-b8e9-287f351be9ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:10.893400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-162c0652-5724-498b-b8e9-287f351be9ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:10.895354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 884/1783] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:10 2022] DELETE /identity/v3/projects/de1996aa222a4670bb5d42d4b8b3aab6 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:10.901006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8779-0bfd-4c51-a3c2-8d065fd299ac None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:10.901509 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8779-0bfd-4c51-a3c2-8d065fd299ac None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:10.901509 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8779-0bfd-4c51-a3c2-8d065fd299ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:10.919807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3ccb8779-0bfd-4c51-a3c2-8d065fd299ac None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:10.921057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3ccb8779-0bfd-4c51-a3c2-8d065fd299ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:10.997335 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 900/1784] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:10.999284 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311ce0c-6f73-4c3f-b03d-140fce76bebd None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:10.999468 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311ce0c-6f73-4c3f-b03d-140fce76bebd None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:11.000398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311ce0c-6f73-4c3f-b03d-140fce76bebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:11.024926 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d311ce0c-6f73-4c3f-b03d-140fce76bebd None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:11.026481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d311ce0c-6f73-4c3f-b03d-140fce76bebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:11.035728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 885/1785] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:11.040259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb8779-0bfd-4c51-a3c2-8d065fd299ac None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:11.050639 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb8779-0bfd-4c51-a3c2-8d065fd299ac None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:11.055724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb702632-82dc-426c-990d-ae6b2c49afa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:11.056584 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb702632-82dc-426c-990d-ae6b2c49afa5 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:11.056862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb702632-82dc-426c-990d-ae6b2c49afa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:11.057135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb702632-82dc-426c-990d-ae6b2c49afa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:11.057914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb702632-82dc-426c-990d-ae6b2c49afa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:11.058216 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb702632-82dc-426c-990d-ae6b2c49afa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:11.059895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb702632-82dc-426c-990d-ae6b2c49afa5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:11.078215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 901/1786] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:11.083961 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311ce0c-6f73-4c3f-b03d-140fce76bebd None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:11.094893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311ce0c-6f73-4c3f-b03d-140fce76bebd None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:11.099974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80bea4-6529-46c8-adde-5c1701fa6d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:11.100741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80bea4-6529-46c8-adde-5c1701fa6d2d admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:11.101084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80bea4-6529-46c8-adde-5c1701fa6d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:11.101422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80bea4-6529-46c8-adde-5c1701fa6d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:11.102176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80bea4-6529-46c8-adde-5c1701fa6d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:11.104199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80bea4-6529-46c8-adde-5c1701fa6d2d admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:11.179521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 886/1787] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:11 2022] POST /identity/v3/projects => generated 370 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:11.183863 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb702632-82dc-426c-990d-ae6b2c49afa5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:11.195154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb702632-82dc-426c-990d-ae6b2c49afa5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:11.254846 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0e5cb-4ea1-49b1-8a2c-aeca5daee11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:11.255580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0e5cb-4ea1-49b1-8a2c-aeca5daee11a admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:11.255856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0e5cb-4ea1-49b1-8a2c-aeca5daee11a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:11.256128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0e5cb-4ea1-49b1-8a2c-aeca5daee11a admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:11.256811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0e5cb-4ea1-49b1-8a2c-aeca5daee11a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:11.258699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0e5cb-4ea1-49b1-8a2c-aeca5daee11a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:11.374226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 902/1788] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:11 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:11.378385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80bea4-6529-46c8-adde-5c1701fa6d2d admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:11.388707 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80bea4-6529-46c8-adde-5c1701fa6d2d admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:11.393277 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db29c29-46fc-4286-a499-538a466c9722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:11.394492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db29c29-46fc-4286-a499-538a466c9722 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:11.394492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db29c29-46fc-4286-a499-538a466c9722 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:11.394492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db29c29-46fc-4286-a499-538a466c9722 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:11.395229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db29c29-46fc-4286-a499-538a466c9722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:11.396843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db29c29-46fc-4286-a499-538a466c9722 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:11.409290 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 887/1789] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:11 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:11.413486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0e5cb-4ea1-49b1-8a2c-aeca5daee11a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:11.424952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0e5cb-4ea1-49b1-8a2c-aeca5daee11a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:11.429640 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6fc115-4574-4871-9097-7eafc45dd585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:11.430312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fc115-4574-4871-9097-7eafc45dd585 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:11.430592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fc115-4574-4871-9097-7eafc45dd585 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:11.434976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fc115-4574-4871-9097-7eafc45dd585 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:11.434976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6fc115-4574-4871-9097-7eafc45dd585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:11.434976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6fc115-4574-4871-9097-7eafc45dd585 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:11.443494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 903/1790] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:11 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:11.447801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db29c29-46fc-4286-a499-538a466c9722 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:11.459338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db29c29-46fc-4286-a499-538a466c9722 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:11.464967 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77bc1a-573c-4770-b749-cbde4dbeed18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:11.465870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77bc1a-573c-4770-b749-cbde4dbeed18 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:11.466147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77bc1a-573c-4770-b749-cbde4dbeed18 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:11.466422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77bc1a-573c-4770-b749-cbde4dbeed18 admin admin] PATH_INFO: `/v3/projects/43345e84926347b0b1f61592243ada62/users/8f7a6a6406d244ea8283e4edab6209aa/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:11.486723 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77bc1a-573c-4770-b749-cbde4dbeed18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43345e84926347b0b1f61592243ada62, user_id=8f7a6a6406d244ea8283e4edab6209aa, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:11.489416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77bc1a-573c-4770-b749-cbde4dbeed18 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:11.568632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 888/1791] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:11 2022] PUT /identity/v3/projects/43345e84926347b0b1f61592243ada62/users/8f7a6a6406d244ea8283e4edab6209aa/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:11.580291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979897d-2bcb-425b-b3f7-bfd1e94f2176 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:11.580587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979897d-2bcb-425b-b3f7-bfd1e94f2176 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:11.580860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979897d-2bcb-425b-b3f7-bfd1e94f2176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:11.617213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b979897d-2bcb-425b-b3f7-bfd1e94f2176 None None] MFA Rules not processed for user `8f7a6a6406d244ea8283e4edab6209aa`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:11.623326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b979897d-2bcb-425b-b3f7-bfd1e94f2176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:11.704562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 904/1792] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:11.711629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77bc1a-573c-4770-b749-cbde4dbeed18 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:11.757303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979897d-2bcb-425b-b3f7-bfd1e94f2176 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:11.759716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b979897d-2bcb-425b-b3f7-bfd1e94f2176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:11.776785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:11.800611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77bc1a-573c-4770-b749-cbde4dbeed18 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:11.806083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e8f35-58fc-4879-8c10-1d2e8aec9624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:11.807408 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e8f35-58fc-4879-8c10-1d2e8aec9624 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:11.807693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e8f35-58fc-4879-8c10-1d2e8aec9624 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:11.807968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e8f35-58fc-4879-8c10-1d2e8aec9624 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:11.808731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e8f35-58fc-4879-8c10-1d2e8aec9624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:11.810612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e8f35-58fc-4879-8c10-1d2e8aec9624 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:11.858759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979897d-2bcb-425b-b3f7-bfd1e94f2176 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:11.864803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76509d-1260-4e7c-a10b-90f415687128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:11.868543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76509d-1260-4e7c-a10b-90f415687128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:11.868543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76509d-1260-4e7c-a10b-90f415687128 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:11.868543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76509d-1260-4e7c-a10b-90f415687128 admin admin] PATH_INFO: `/v3/users/da9769988c8a474099ad45f7e006a8ef` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:11.868543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b76509d-1260-4e7c-a10b-90f415687128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da9769988c8a474099ad45f7e006a8ef)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:11.869422 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b76509d-1260-4e7c-a10b-90f415687128 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:11.961256 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 889/1793] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:11 2022] POST /identity/v3/projects => generated 368 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:11.965298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e8f35-58fc-4879-8c10-1d2e8aec9624 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:12.045987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e8f35-58fc-4879-8c10-1d2e8aec9624 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:12.116012 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59747c67-6b1e-4c60-a039-cc1cca91b0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:12.116012 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59747c67-6b1e-4c60-a039-cc1cca91b0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:12.116012 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59747c67-6b1e-4c60-a039-cc1cca91b0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:12.116012 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59747c67-6b1e-4c60-a039-cc1cca91b0f9 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:12.118022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59747c67-6b1e-4c60-a039-cc1cca91b0f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:12.118022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59747c67-6b1e-4c60-a039-cc1cca91b0f9 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:12.965454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 890/1794] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:11 2022] POST /identity/v3/users => generated 291 bytes in 1000 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:12.969644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b76509d-1260-4e7c-a10b-90f415687128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da9769988c8a474099ad45f7e006a8ef'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:12.971644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59747c67-6b1e-4c60-a039-cc1cca91b0f9 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:12.976911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b76509d-1260-4e7c-a10b-90f415687128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da9769988c8a474099ad45f7e006a8ef'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:12.977630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b76509d-1260-4e7c-a10b-90f415687128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da9769988c8a474099ad45f7e006a8ef'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:12.987653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59747c67-6b1e-4c60-a039-cc1cca91b0f9 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:12.991626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b76509d-1260-4e7c-a10b-90f415687128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da9769988c8a474099ad45f7e006a8ef'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:13.004971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5ff4c-1290-4964-827d-51f3774e90ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:13.005665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5ff4c-1290-4964-827d-51f3774e90ab admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:13.005945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5ff4c-1290-4964-827d-51f3774e90ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:13.006354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5ff4c-1290-4964-827d-51f3774e90ab admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:13.006992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5ff4c-1290-4964-827d-51f3774e90ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:13.008629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5ff4c-1290-4964-827d-51f3774e90ab admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:13.023747 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 891/1795] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:12 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:13.024808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5ff4c-1290-4964-827d-51f3774e90ab admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:13.039704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5ff4c-1290-4964-827d-51f3774e90ab admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:13.045109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188bad7-448e-41c0-8551-6ec27dbde4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:13.054285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188bad7-448e-41c0-8551-6ec27dbde4fa admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:13.054285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188bad7-448e-41c0-8551-6ec27dbde4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:13.054285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188bad7-448e-41c0-8551-6ec27dbde4fa admin admin] PATH_INFO: `/v3/projects/e52436b2131246b7bc2bcd0800ac0b7a/users/9649e47995b644bca6ca9fad5d836441/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:13.068262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188bad7-448e-41c0-8551-6ec27dbde4fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e52436b2131246b7bc2bcd0800ac0b7a, user_id=9649e47995b644bca6ca9fad5d836441, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:13.071250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188bad7-448e-41c0-8551-6ec27dbde4fa admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:13.098938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b76509d-1260-4e7c-a10b-90f415687128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da9769988c8a474099ad45f7e006a8ef'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:13.106200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 905/1796] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:11 2022] DELETE /identity/v3/users/da9769988c8a474099ad45f7e006a8ef => generated 0 bytes in 1349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:13.111912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496b486-65e9-4018-8ca9-609b68f38086 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:13.112334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496b486-65e9-4018-8ca9-609b68f38086 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:13.112550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496b486-65e9-4018-8ca9-609b68f38086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:13.137257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c496b486-65e9-4018-8ca9-609b68f38086 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:13.138637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c496b486-65e9-4018-8ca9-609b68f38086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:13.165483 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 892/1797] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:13 2022] PUT /identity/v3/projects/e52436b2131246b7bc2bcd0800ac0b7a/users/9649e47995b644bca6ca9fad5d836441/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:13.166597 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188bad7-448e-41c0-8551-6ec27dbde4fa admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:13.176469 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a188bad7-448e-41c0-8551-6ec27dbde4fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:13.234243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 906/1798] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:13.253975 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496b486-65e9-4018-8ca9-609b68f38086 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:13.260179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c496b486-65e9-4018-8ca9-609b68f38086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:13.262299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188bad7-448e-41c0-8551-6ec27dbde4fa admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:13.267968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba55951-40b5-44d3-bc09-a82335106321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:13.268786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba55951-40b5-44d3-bc09-a82335106321 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:13.269162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba55951-40b5-44d3-bc09-a82335106321 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:13.269538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba55951-40b5-44d3-bc09-a82335106321 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:13.270368 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba55951-40b5-44d3-bc09-a82335106321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:13.272114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba55951-40b5-44d3-bc09-a82335106321 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:13.286919 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 893/1799] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:13 2022] GET /identity/v3/roles => generated 1427 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:13.290755 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba55951-40b5-44d3-bc09-a82335106321 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:13.298239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba55951-40b5-44d3-bc09-a82335106321 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:13.302871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00198aa-74df-4ade-89c6-b3b00db5306c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:13.304007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00198aa-74df-4ade-89c6-b3b00db5306c admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:13.304913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00198aa-74df-4ade-89c6-b3b00db5306c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:13.304913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00198aa-74df-4ade-89c6-b3b00db5306c admin admin] PATH_INFO: `/v3/projects/e52436b2131246b7bc2bcd0800ac0b7a/users/9649e47995b644bca6ca9fad5d836441/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:13.306533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00198aa-74df-4ade-89c6-b3b00db5306c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e52436b2131246b7bc2bcd0800ac0b7a, user_id=9649e47995b644bca6ca9fad5d836441, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:13.309304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00198aa-74df-4ade-89c6-b3b00db5306c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:13.320665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496b486-65e9-4018-8ca9-609b68f38086 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:13.321039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 894/1800] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:13 2022] PUT /identity/v3/projects/e52436b2131246b7bc2bcd0800ac0b7a/users/9649e47995b644bca6ca9fad5d836441/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:13.332070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c401165-7339-42c0-8482-8e232d1d8c5a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:13.332358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c401165-7339-42c0-8482-8e232d1d8c5a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:13.332633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c401165-7339-42c0-8482-8e232d1d8c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:13.360610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-7c401165-7339-42c0-8482-8e232d1d8c5a None None] MFA Rules not processed for user `9649e47995b644bca6ca9fad5d836441`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:13.361816 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7c401165-7339-42c0-8482-8e232d1d8c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:13.377658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130479a8-aa78-4dd6-8e46-2fb1875aeec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:13.378386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130479a8-aa78-4dd6-8e46-2fb1875aeec5 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:13.378677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130479a8-aa78-4dd6-8e46-2fb1875aeec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:13.378985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130479a8-aa78-4dd6-8e46-2fb1875aeec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:13.451027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130479a8-aa78-4dd6-8e46-2fb1875aeec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:13.452472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130479a8-aa78-4dd6-8e46-2fb1875aeec5 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:13.454633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 895/1801] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:13.455522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c401165-7339-42c0-8482-8e232d1d8c5a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:13.461837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7c401165-7339-42c0-8482-8e232d1d8c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:13.465299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 907/1802] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:13.470900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c401165-7339-42c0-8482-8e232d1d8c5a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:13.474326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130479a8-aa78-4dd6-8e46-2fb1875aeec5 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:13.475829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:13.476026 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-130479a8-aa78-4dd6-8e46-2fb1875aeec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:13.476927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:13.477210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:13.477485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] PATH_INFO: `/v3/users/add6b90e58e943579c0f152f25a77e2a` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:13.478727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=add6b90e58e943579c0f152f25a77e2a)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:13.480850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:13.540784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130479a8-aa78-4dd6-8e46-2fb1875aeec5 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:13.544753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82eef0-b621-484c-932d-d22204fcfcdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:13.545420 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82eef0-b621-484c-932d-d22204fcfcdd service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:13.545731 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82eef0-b621-484c-932d-d22204fcfcdd service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:13.546041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82eef0-b621-484c-932d-d22204fcfcdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:13.559074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82eef0-b621-484c-932d-d22204fcfcdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:13.559656 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82eef0-b621-484c-932d-d22204fcfcdd service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:13.570975 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 908/1803] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:13.720773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add6b90e58e943579c0f152f25a77e2a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:13.731168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add6b90e58e943579c0f152f25a77e2a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:13.735788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add6b90e58e943579c0f152f25a77e2a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:13.747644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add6b90e58e943579c0f152f25a77e2a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:13.754679 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:13.847811 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add6b90e58e943579c0f152f25a77e2a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:13.854909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 896/1804] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:13 2022] DELETE /identity/v3/users/add6b90e58e943579c0f152f25a77e2a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:13.859462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf90a9a-b990-4fa3-b01d-be9da5e07985 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:13.860011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf90a9a-b990-4fa3-b01d-be9da5e07985 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:13.860392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf90a9a-b990-4fa3-b01d-be9da5e07985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:13.879333 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ddf90a9a-b990-4fa3-b01d-be9da5e07985 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:13.880594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ddf90a9a-b990-4fa3-b01d-be9da5e07985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:13.945561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:13.947285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:13.951877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 909/1805] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:13.964495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf90a9a-b990-4fa3-b01d-be9da5e07985 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:13.967176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ddf90a9a-b990-4fa3-b01d-be9da5e07985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:13.985743 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1f1bc-7118-4c91-a22d-c0c4e4696697 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:13.991570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:13.993871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:13.993871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:13.993871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] PATH_INFO: `/v3/projects/6693c9fed829462b95fb8db250d95ccc` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:13.995888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6693c9fed829462b95fb8db250d95ccc)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:13.998246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:14.065955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf90a9a-b990-4fa3-b01d-be9da5e07985 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:14.070321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93d3f1-7938-429f-8606-f3fd9dafd19c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:14.071238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93d3f1-7938-429f-8606-f3fd9dafd19c service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:14.071436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93d3f1-7938-429f-8606-f3fd9dafd19c service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:14.078735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93d3f1-7938-429f-8606-f3fd9dafd19c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:14.084001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93d3f1-7938-429f-8606-f3fd9dafd19c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:14.085449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93d3f1-7938-429f-8606-f3fd9dafd19c service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:14.097321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 910/1806] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:14.153178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6693c9fed829462b95fb8db250d95ccc', 'request_id': 'req-3859d0de-5515-4ac3-860d-721b6dc56204'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.283266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6693c9fed829462b95fb8db250d95ccc', 'request_id': 'req-3859d0de-5515-4ac3-860d-721b6dc56204'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.291073 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6693c9fed829462b95fb8db250d95ccc', 'request_id': 'req-3859d0de-5515-4ac3-860d-721b6dc56204'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.302695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] The token cache is being invalidate because project 6693c9fed829462b95fb8db250d95ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:14.303099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.304113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.305817 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 897/1807] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:13 2022] DELETE /identity/v3/projects/6693c9fed829462b95fb8db250d95ccc => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:14.309963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93d3f1-7938-429f-8606-f3fd9dafd19c service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:14.320792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93d3f1-7938-429f-8606-f3fd9dafd19c service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:14.451929 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:14.453922 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:14.454445 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:14.454977 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] PATH_INFO: `/v3/users/72b560db4df84b47b033f89cad737b0f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:14.456998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b560db4df84b47b033f89cad737b0f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:14.460999 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:14.633194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:14.635704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:14.639372 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:14.655100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859d0de-5515-4ac3-860d-721b6dc56204 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:14.662074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b560db4df84b47b033f89cad737b0f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.671318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:14.673104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:14.673104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:14.673444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] PATH_INFO: `/v3/projects/f44ac7096ce24241ac9984676d801b52` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:14.676038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f44ac7096ce24241ac9984676d801b52)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:14.677708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b560db4df84b47b033f89cad737b0f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.678298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:14.678453 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b560db4df84b47b033f89cad737b0f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.694410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b560db4df84b47b033f89cad737b0f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.768839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b560db4df84b47b033f89cad737b0f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.775893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 911/1808] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:14 2022] DELETE /identity/v3/users/72b560db4df84b47b033f89cad737b0f => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:14.812535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f44ac7096ce24241ac9984676d801b52', 'request_id': 'req-812eca27-8507-46ca-a245-504924bbdd19'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.865808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:14.918137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f44ac7096ce24241ac9984676d801b52', 'request_id': 'req-812eca27-8507-46ca-a245-504924bbdd19'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.925273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f44ac7096ce24241ac9984676d801b52', 'request_id': 'req-812eca27-8507-46ca-a245-504924bbdd19'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.936413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] The token cache is being invalidate because project f44ac7096ce24241ac9984676d801b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:14.936701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.938043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:14.940179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 898/1809] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:14 2022] DELETE /identity/v3/projects/f44ac7096ce24241ac9984676d801b52 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:14.944379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:14.945848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:15.020035 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953d380-e1d9-464c-993d-c979e5803f87 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:15.084923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715318e-eba6-4248-a6b0-f3126aaf6ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:15.085735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715318e-eba6-4248-a6b0-f3126aaf6ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:15.086014 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715318e-eba6-4248-a6b0-f3126aaf6ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:15.086334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715318e-eba6-4248-a6b0-f3126aaf6ba5 admin admin] PATH_INFO: `/v3/users/61a17846d01f4107a6aaa60c876bd26f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:15.089057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715318e-eba6-4248-a6b0-f3126aaf6ba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a17846d01f4107a6aaa60c876bd26f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:15.091033 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715318e-eba6-4248-a6b0-f3126aaf6ba5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:15.249804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1715318e-eba6-4248-a6b0-f3126aaf6ba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a17846d01f4107a6aaa60c876bd26f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:15.257231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1715318e-eba6-4248-a6b0-f3126aaf6ba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a17846d01f4107a6aaa60c876bd26f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:15.258170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1715318e-eba6-4248-a6b0-f3126aaf6ba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a17846d01f4107a6aaa60c876bd26f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:15.266308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:15.271401 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:15.292846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812eca27-8507-46ca-a245-504924bbdd19 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:15.311872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1715318e-eba6-4248-a6b0-f3126aaf6ba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a17846d01f4107a6aaa60c876bd26f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:15.312198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee37b6-fea9-4bc1-a543-35442d6e48b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:15.312198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee37b6-fea9-4bc1-a543-35442d6e48b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:15.312198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee37b6-fea9-4bc1-a543-35442d6e48b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:15.312198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee37b6-fea9-4bc1-a543-35442d6e48b4 admin admin] PATH_INFO: `/v3/projects/2ce08c459e734bdc81d35996f3cd8274` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:15.312502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee37b6-fea9-4bc1-a543-35442d6e48b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce08c459e734bdc81d35996f3cd8274)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:15.312502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee37b6-fea9-4bc1-a543-35442d6e48b4 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:15.399992 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1715318e-eba6-4248-a6b0-f3126aaf6ba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a17846d01f4107a6aaa60c876bd26f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:15.411684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 912/1810] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:14 2022] DELETE /identity/v3/users/61a17846d01f4107a6aaa60c876bd26f => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:15.471218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-20ee37b6-fea9-4bc1-a543-35442d6e48b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce08c459e734bdc81d35996f3cd8274', 'request_id': 'req-20ee37b6-fea9-4bc1-a543-35442d6e48b4'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:15.549767 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-20ee37b6-fea9-4bc1-a543-35442d6e48b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce08c459e734bdc81d35996f3cd8274', 'request_id': 'req-20ee37b6-fea9-4bc1-a543-35442d6e48b4'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:15.561486 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-20ee37b6-fea9-4bc1-a543-35442d6e48b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce08c459e734bdc81d35996f3cd8274', 'request_id': 'req-20ee37b6-fea9-4bc1-a543-35442d6e48b4'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:15.573951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-20ee37b6-fea9-4bc1-a543-35442d6e48b4 admin admin] The token cache is being invalidate because project 2ce08c459e734bdc81d35996f3cd8274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:15.574240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-20ee37b6-fea9-4bc1-a543-35442d6e48b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:15.574981 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-20ee37b6-fea9-4bc1-a543-35442d6e48b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:15.581140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 899/1811] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:15 2022] DELETE /identity/v3/projects/2ce08c459e734bdc81d35996f3cd8274 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:15.585887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304f1e4-1a3f-461f-b863-705bb0b44ed9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:15.586191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304f1e4-1a3f-461f-b863-705bb0b44ed9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:15.586633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304f1e4-1a3f-461f-b863-705bb0b44ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:15.609880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8304f1e4-1a3f-461f-b863-705bb0b44ed9 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:15.612025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8304f1e4-1a3f-461f-b863-705bb0b44ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:15.681952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 913/1812] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:15.690334 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d986439-df68-4330-b03b-ce3ac575bd0f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:15.690628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d986439-df68-4330-b03b-ce3ac575bd0f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:15.690946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d986439-df68-4330-b03b-ce3ac575bd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:15.712667 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5d986439-df68-4330-b03b-ce3ac575bd0f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:15.713896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5d986439-df68-4330-b03b-ce3ac575bd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:15.724235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 900/1813] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:15.728260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304f1e4-1a3f-461f-b863-705bb0b44ed9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:15.739038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304f1e4-1a3f-461f-b863-705bb0b44ed9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:15.743950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9f9a7-bbb5-4bf5-9f4b-eaea3f901292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:15.747162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9f9a7-bbb5-4bf5-9f4b-eaea3f901292 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:15.747162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9f9a7-bbb5-4bf5-9f4b-eaea3f901292 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:15.747162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9f9a7-bbb5-4bf5-9f4b-eaea3f901292 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:15.747162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9f9a7-bbb5-4bf5-9f4b-eaea3f901292 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:15.747162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9f9a7-bbb5-4bf5-9f4b-eaea3f901292 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:15.747796 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9f9a7-bbb5-4bf5-9f4b-eaea3f901292 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:15.760837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 914/1814] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:15.765182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d986439-df68-4330-b03b-ce3ac575bd0f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:15.776019 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d986439-df68-4330-b03b-ce3ac575bd0f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:15.780824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34c1b3-76ee-4ae3-8eaa-6636fc49e235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:15.781578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34c1b3-76ee-4ae3-8eaa-6636fc49e235 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:15.781867 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34c1b3-76ee-4ae3-8eaa-6636fc49e235 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:15.782141 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34c1b3-76ee-4ae3-8eaa-6636fc49e235 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:15.783716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34c1b3-76ee-4ae3-8eaa-6636fc49e235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:15.786010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34c1b3-76ee-4ae3-8eaa-6636fc49e235 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:15.905246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 901/1815] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:15 2022] POST /identity/v3/projects => generated 374 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:15.906384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9f9a7-bbb5-4bf5-9f4b-eaea3f901292 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:15.918853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9f9a7-bbb5-4bf5-9f4b-eaea3f901292 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:15.989439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186cfed-af2c-42e9-8690-0cf27ef6ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:15.990125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186cfed-af2c-42e9-8690-0cf27ef6ca23 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:15.990403 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186cfed-af2c-42e9-8690-0cf27ef6ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:15.990676 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186cfed-af2c-42e9-8690-0cf27ef6ca23 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:15.991465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186cfed-af2c-42e9-8690-0cf27ef6ca23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:15.993302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186cfed-af2c-42e9-8690-0cf27ef6ca23 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:16.033674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34c1b3-76ee-4ae3-8eaa-6636fc49e235 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:16.035526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ea34c1b3-76ee-4ae3-8eaa-6636fc49e235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:16.053539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34c1b3-76ee-4ae3-8eaa-6636fc49e235 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:16.058267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d10016-4f2a-428d-8a3a-2522fd555e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:16.059179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d10016-4f2a-428d-8a3a-2522fd555e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:16.059470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d10016-4f2a-428d-8a3a-2522fd555e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:16.060031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d10016-4f2a-428d-8a3a-2522fd555e67 admin admin] PATH_INFO: `/v3/projects/81ca2bbff5914bed90929efd7e41faf9` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:16.061188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d10016-4f2a-428d-8a3a-2522fd555e67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ca2bbff5914bed90929efd7e41faf9)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:16.063785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d10016-4f2a-428d-8a3a-2522fd555e67 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:16.118263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 915/1816] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:15 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:16.122071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186cfed-af2c-42e9-8690-0cf27ef6ca23 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:16.135694 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186cfed-af2c-42e9-8690-0cf27ef6ca23 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:16.141765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b97867-af7e-48d3-b981-5ce355f307a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:16.145307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b97867-af7e-48d3-b981-5ce355f307a3 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:16.145307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b97867-af7e-48d3-b981-5ce355f307a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:16.145307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b97867-af7e-48d3-b981-5ce355f307a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:16.145307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b97867-af7e-48d3-b981-5ce355f307a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:16.145914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b97867-af7e-48d3-b981-5ce355f307a3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:16.158727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 916/1817] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:16 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:16.163077 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b97867-af7e-48d3-b981-5ce355f307a3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:16.176160 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b97867-af7e-48d3-b981-5ce355f307a3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:16.182307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7592074-4036-4724-bd7b-da881d16290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:16.184703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7592074-4036-4724-bd7b-da881d16290c admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:16.185018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7592074-4036-4724-bd7b-da881d16290c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:16.185339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7592074-4036-4724-bd7b-da881d16290c admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:16.186066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7592074-4036-4724-bd7b-da881d16290c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:16.187893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7592074-4036-4724-bd7b-da881d16290c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:16.209781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 917/1818] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:16 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:16.210919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7592074-4036-4724-bd7b-da881d16290c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:16.216040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a0d10016-4f2a-428d-8a3a-2522fd555e67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ca2bbff5914bed90929efd7e41faf9', 'request_id': 'req-a0d10016-4f2a-428d-8a3a-2522fd555e67'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:16.235706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7592074-4036-4724-bd7b-da881d16290c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:16.314649 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a0d10016-4f2a-428d-8a3a-2522fd555e67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ca2bbff5914bed90929efd7e41faf9', 'request_id': 'req-a0d10016-4f2a-428d-8a3a-2522fd555e67'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:16.319431 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:16.332917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a0d10016-4f2a-428d-8a3a-2522fd555e67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ca2bbff5914bed90929efd7e41faf9', 'request_id': 'req-a0d10016-4f2a-428d-8a3a-2522fd555e67'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:16.359647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd0c09-5d12-4d80-ac00-9c48a832a4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:16.372166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd0c09-5d12-4d80-ac00-9c48a832a4a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:16.373246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd0c09-5d12-4d80-ac00-9c48a832a4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:16.373716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd0c09-5d12-4d80-ac00-9c48a832a4a5 admin admin] PATH_INFO: `/v3/projects/5629e91c32b04a4a9bd193a0c8728274/users/4408a4b02d2a4f56ae17f1ec5c23bd8a/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:16.386322 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a0d10016-4f2a-428d-8a3a-2522fd555e67 admin admin] The token cache is being invalidate because project 81ca2bbff5914bed90929efd7e41faf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:16.386794 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a0d10016-4f2a-428d-8a3a-2522fd555e67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:16.387614 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a0d10016-4f2a-428d-8a3a-2522fd555e67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:16.390869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 902/1819] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:16 2022] DELETE /identity/v3/projects/81ca2bbff5914bed90929efd7e41faf9 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:16.396512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ccbe3-e8ee-466a-a2bd-cb5582e776c1 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:16.397044 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ccbe3-e8ee-466a-a2bd-cb5582e776c1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:16.397550 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ccbe3-e8ee-466a-a2bd-cb5582e776c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:16.400626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd0c09-5d12-4d80-ac00-9c48a832a4a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5629e91c32b04a4a9bd193a0c8728274, user_id=4408a4b02d2a4f56ae17f1ec5c23bd8a, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:16.409278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd0c09-5d12-4d80-ac00-9c48a832a4a5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:16.423180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-306ccbe3-e8ee-466a-a2bd-cb5582e776c1 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:16.425257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-306ccbe3-e8ee-466a-a2bd-cb5582e776c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:16.433726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 903/1820] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:16.441467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618705e-9b36-4342-911a-c034d4c032e5 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:16.442756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618705e-9b36-4342-911a-c034d4c032e5 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:16.443138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618705e-9b36-4342-911a-c034d4c032e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:16.464060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f618705e-9b36-4342-911a-c034d4c032e5 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:16.465365 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f618705e-9b36-4342-911a-c034d4c032e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:16.478701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 904/1821] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:16.480150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618705e-9b36-4342-911a-c034d4c032e5 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:16.494521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618705e-9b36-4342-911a-c034d4c032e5 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:16.502403 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 918/1822] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:16 2022] PUT /identity/v3/projects/5629e91c32b04a4a9bd193a0c8728274/users/4408a4b02d2a4f56ae17f1ec5c23bd8a/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:16.507899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073020fa-41c0-4285-bdec-9a2fb7058a36 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:16.508084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073020fa-41c0-4285-bdec-9a2fb7058a36 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:16.508458 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073020fa-41c0-4285-bdec-9a2fb7058a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:16.539451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-073020fa-41c0-4285-bdec-9a2fb7058a36 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:16.542062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-073020fa-41c0-4285-bdec-9a2fb7058a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:16.583846 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:16.589199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bdb1c-a0d2-41bf-8831-4fcc13b514bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:16.589199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bdb1c-a0d2-41bf-8831-4fcc13b514bc admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:16.589199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bdb1c-a0d2-41bf-8831-4fcc13b514bc admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:16.589199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bdb1c-a0d2-41bf-8831-4fcc13b514bc admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:16.589527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bdb1c-a0d2-41bf-8831-4fcc13b514bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:16.589527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bdb1c-a0d2-41bf-8831-4fcc13b514bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:16.591117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bdb1c-a0d2-41bf-8831-4fcc13b514bc admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:16.611633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 905/1823] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:16.615729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bdb1c-a0d2-41bf-8831-4fcc13b514bc admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:16.627485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bdb1c-a0d2-41bf-8831-4fcc13b514bc admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:16.632495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27f3e9-ec7a-483a-b40e-4f88a23c347b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:16.633345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27f3e9-ec7a-483a-b40e-4f88a23c347b admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:16.633633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27f3e9-ec7a-483a-b40e-4f88a23c347b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:16.633910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27f3e9-ec7a-483a-b40e-4f88a23c347b admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:16.634861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27f3e9-ec7a-483a-b40e-4f88a23c347b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:16.637160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27f3e9-ec7a-483a-b40e-4f88a23c347b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:16.643113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 919/1824] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:16.656869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073020fa-41c0-4285-bdec-9a2fb7058a36 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:16.659326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-073020fa-41c0-4285-bdec-9a2fb7058a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:16.737882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073020fa-41c0-4285-bdec-9a2fb7058a36 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:16.740726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 906/1825] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:16 2022] POST /identity/v3/projects => generated 386 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:16.746848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27f3e9-ec7a-483a-b40e-4f88a23c347b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:16.822004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54c596-3c53-41a9-8375-e10db372b96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:16.823931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54c596-3c53-41a9-8375-e10db372b96d service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:16.825136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54c596-3c53-41a9-8375-e10db372b96d service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:16.825779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54c596-3c53-41a9-8375-e10db372b96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:16.869106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27f3e9-ec7a-483a-b40e-4f88a23c347b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:16.879817 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efc11b-fa03-4960-9195-5a53537222d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:16.881185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efc11b-fa03-4960-9195-5a53537222d7 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:16.881682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efc11b-fa03-4960-9195-5a53537222d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:16.882275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efc11b-fa03-4960-9195-5a53537222d7 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:16.883509 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efc11b-fa03-4960-9195-5a53537222d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:16.887169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efc11b-fa03-4960-9195-5a53537222d7 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:16.944263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54c596-3c53-41a9-8375-e10db372b96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:16.949063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54c596-3c53-41a9-8375-e10db372b96d service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:16.960300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 920/1826] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:17.008295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 907/1827] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:16 2022] POST /identity/v3/users => generated 294 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:17.012644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54c596-3c53-41a9-8375-e10db372b96d service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:17.020158 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:17.026680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54c596-3c53-41a9-8375-e10db372b96d service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:17.031654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457eec50-bb13-44f5-998b-713cc146fabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:17.033034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457eec50-bb13-44f5-998b-713cc146fabc admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:17.033372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457eec50-bb13-44f5-998b-713cc146fabc admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:17.033658 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457eec50-bb13-44f5-998b-713cc146fabc admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:17.034471 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457eec50-bb13-44f5-998b-713cc146fabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:17.036127 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457eec50-bb13-44f5-998b-713cc146fabc admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:17.047445 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 921/1828] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:17 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:17.051567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efc11b-fa03-4960-9195-5a53537222d7 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:17.068749 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efc11b-fa03-4960-9195-5a53537222d7 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:17.078238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f748230-10ed-4849-a76b-ef89ccb21127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:17.078976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f748230-10ed-4849-a76b-ef89ccb21127 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:17.079796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f748230-10ed-4849-a76b-ef89ccb21127 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:17.080024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f748230-10ed-4849-a76b-ef89ccb21127 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:17.080766 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f748230-10ed-4849-a76b-ef89ccb21127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:17.082509 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f748230-10ed-4849-a76b-ef89ccb21127 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:17.100920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 908/1829] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:17 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:17.105205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457eec50-bb13-44f5-998b-713cc146fabc admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:17.108205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f748230-10ed-4849-a76b-ef89ccb21127 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:17.111054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-2f748230-10ed-4849-a76b-ef89ccb21127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:17.117260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457eec50-bb13-44f5-998b-713cc146fabc admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:17.121965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecc2a0-afaa-4346-9171-4e90b5b70372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:17.122904 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecc2a0-afaa-4346-9171-4e90b5b70372 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:17.123180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecc2a0-afaa-4346-9171-4e90b5b70372 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:17.123460 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecc2a0-afaa-4346-9171-4e90b5b70372 admin admin] PATH_INFO: `/v3/projects/0964236d544a4144b7512a8867810dc1/users/0f60a274b1c14cef973be40b02126463/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:17.128867 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f748230-10ed-4849-a76b-ef89ccb21127 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:17.134776 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c59ba1-d77f-42c6-b956-621e565373c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:17.135701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c59ba1-d77f-42c6-b956-621e565373c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:17.135996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c59ba1-d77f-42c6-b956-621e565373c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:17.136257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c59ba1-d77f-42c6-b956-621e565373c5 admin admin] PATH_INFO: `/v3/users/8f7a6a6406d244ea8283e4edab6209aa` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:17.137991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c59ba1-d77f-42c6-b956-621e565373c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f7a6a6406d244ea8283e4edab6209aa)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:17.140234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c59ba1-d77f-42c6-b956-621e565373c5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:17.147862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecc2a0-afaa-4346-9171-4e90b5b70372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0964236d544a4144b7512a8867810dc1, user_id=0f60a274b1c14cef973be40b02126463, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:17.150941 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecc2a0-afaa-4346-9171-4e90b5b70372 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:17.228773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 922/1830] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:17 2022] PUT /identity/v3/projects/0964236d544a4144b7512a8867810dc1/users/0f60a274b1c14cef973be40b02126463/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:17.239485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192e31f-0058-4058-9cb2-4cdfc5a812fc None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:17.239787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192e31f-0058-4058-9cb2-4cdfc5a812fc None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:17.240065 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192e31f-0058-4058-9cb2-4cdfc5a812fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:17.268854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c192e31f-0058-4058-9cb2-4cdfc5a812fc None None] MFA Rules not processed for user `0f60a274b1c14cef973be40b02126463`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:17.270175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c192e31f-0058-4058-9cb2-4cdfc5a812fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:17.342684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:17.348468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 923/1831] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:17.353200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192e31f-0058-4058-9cb2-4cdfc5a812fc None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:17.418566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-91c59ba1-d77f-42c6-b956-621e565373c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f7a6a6406d244ea8283e4edab6209aa'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:17.426226 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-91c59ba1-d77f-42c6-b956-621e565373c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f7a6a6406d244ea8283e4edab6209aa'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:17.427040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-91c59ba1-d77f-42c6-b956-621e565373c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f7a6a6406d244ea8283e4edab6209aa'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:17.429203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192e31f-0058-4058-9cb2-4cdfc5a812fc None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:17.437718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755f975-2323-4a3f-a741-9c435ad281e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:17.438402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755f975-2323-4a3f-a741-9c435ad281e4 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:17.439292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755f975-2323-4a3f-a741-9c435ad281e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:17.439888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755f975-2323-4a3f-a741-9c435ad281e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:17.440833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755f975-2323-4a3f-a741-9c435ad281e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:17.442963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755f975-2323-4a3f-a741-9c435ad281e4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:17.445014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-91c59ba1-d77f-42c6-b956-621e565373c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f7a6a6406d244ea8283e4edab6209aa'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:17.544185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 924/1832] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:17 2022] POST /identity/v3/projects => generated 388 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:17.556170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755f975-2323-4a3f-a741-9c435ad281e4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:17.663241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755f975-2323-4a3f-a741-9c435ad281e4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:17.665564 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-91c59ba1-d77f-42c6-b956-621e565373c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f7a6a6406d244ea8283e4edab6209aa'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:17.673616 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 909/1833] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:17 2022] DELETE /identity/v3/users/8f7a6a6406d244ea8283e4edab6209aa => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:17.678296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15c2fa-b4cb-40f9-a642-7b68ed042793 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:17.684286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15c2fa-b4cb-40f9-a642-7b68ed042793 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:17.684286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15c2fa-b4cb-40f9-a642-7b68ed042793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:17.685454 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598788c-28fe-489c-8bbf-9884b23a9dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:17.686302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598788c-28fe-489c-8bbf-9884b23a9dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:17.686585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598788c-28fe-489c-8bbf-9884b23a9dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:17.687554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598788c-28fe-489c-8bbf-9884b23a9dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:17.691425 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598788c-28fe-489c-8bbf-9884b23a9dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:17.693474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598788c-28fe-489c-8bbf-9884b23a9dd6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:17.742284 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-9a15c2fa-b4cb-40f9-a642-7b68ed042793 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:17.744418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9a15c2fa-b4cb-40f9-a642-7b68ed042793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:17.811514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 910/1834] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:17.833018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 925/1835] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:17 2022] POST /identity/v3/users => generated 301 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:17.839966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15c2fa-b4cb-40f9-a642-7b68ed042793 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:17.842206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9a15c2fa-b4cb-40f9-a642-7b68ed042793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:17.844136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598788c-28fe-489c-8bbf-9884b23a9dd6 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:17.853877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4598788c-28fe-489c-8bbf-9884b23a9dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:17.862735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15c2fa-b4cb-40f9-a642-7b68ed042793 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:17.870397 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba0623-8e68-4561-a4da-034e558563df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:17.871836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba0623-8e68-4561-a4da-034e558563df admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:17.872820 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba0623-8e68-4561-a4da-034e558563df admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:17.872820 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba0623-8e68-4561-a4da-034e558563df admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:17.874178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba0623-8e68-4561-a4da-034e558563df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:17.877977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba0623-8e68-4561-a4da-034e558563df admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:17.901520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 911/1836] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:17 2022] GET /identity/v3/roles => generated 1427 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:17.902895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba0623-8e68-4561-a4da-034e558563df admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:17.919006 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba0623-8e68-4561-a4da-034e558563df admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:17.924566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0bc84-e647-4ec4-9ae1-5699fad85a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:17.925504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0bc84-e647-4ec4-9ae1-5699fad85a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:17.925785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0bc84-e647-4ec4-9ae1-5699fad85a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:17.926056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0bc84-e647-4ec4-9ae1-5699fad85a3a admin admin] PATH_INFO: `/v3/projects/536360f5d9834fc6aa51e85c331745f4/users/f3199319c15c4ef3bdad507d18726bb9/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:17.946095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0bc84-e647-4ec4-9ae1-5699fad85a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=536360f5d9834fc6aa51e85c331745f4, user_id=f3199319c15c4ef3bdad507d18726bb9, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:17.947505 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0bc84-e647-4ec4-9ae1-5699fad85a3a admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:17.948077 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598788c-28fe-489c-8bbf-9884b23a9dd6 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:18.006526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edf1dc-bbf1-49f0-92d1-4dcbf4b569a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:18.007368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edf1dc-bbf1-49f0-92d1-4dcbf4b569a2 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:18.007776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edf1dc-bbf1-49f0-92d1-4dcbf4b569a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:18.008601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edf1dc-bbf1-49f0-92d1-4dcbf4b569a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:18.019494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edf1dc-bbf1-49f0-92d1-4dcbf4b569a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:18.021077 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edf1dc-bbf1-49f0-92d1-4dcbf4b569a2 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:18.030194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 912/1837] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:17 2022] PUT /identity/v3/projects/536360f5d9834fc6aa51e85c331745f4/users/f3199319c15c4ef3bdad507d18726bb9/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:18.034105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 926/1838] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:18.034615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0bc84-e647-4ec4-9ae1-5699fad85a3a admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:18.194536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0bc84-e647-4ec4-9ae1-5699fad85a3a admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:18.208097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c988be-9b5b-459c-9a7d-04cffb5e8a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:18.210037 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c988be-9b5b-459c-9a7d-04cffb5e8a65 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:18.210453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c988be-9b5b-459c-9a7d-04cffb5e8a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:18.210855 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c988be-9b5b-459c-9a7d-04cffb5e8a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:18.212684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c988be-9b5b-459c-9a7d-04cffb5e8a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:18.216049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c988be-9b5b-459c-9a7d-04cffb5e8a65 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:18.233611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 913/1839] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:18 2022] GET /identity/v3/roles => generated 1427 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:18.239238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edf1dc-bbf1-49f0-92d1-4dcbf4b569a2 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:18.255542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edf1dc-bbf1-49f0-92d1-4dcbf4b569a2 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:18.262515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed91e8-72d4-44ed-b049-c2f82ee813e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:18.265199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed91e8-72d4-44ed-b049-c2f82ee813e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:18.265694 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed91e8-72d4-44ed-b049-c2f82ee813e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:18.266569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed91e8-72d4-44ed-b049-c2f82ee813e2 admin admin] PATH_INFO: `/v3/projects/536360f5d9834fc6aa51e85c331745f4/users/f3199319c15c4ef3bdad507d18726bb9/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:18.270341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed91e8-72d4-44ed-b049-c2f82ee813e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=536360f5d9834fc6aa51e85c331745f4, user_id=f3199319c15c4ef3bdad507d18726bb9, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:18.279683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed91e8-72d4-44ed-b049-c2f82ee813e2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:18.307530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 927/1840] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:18 2022] PUT /identity/v3/projects/536360f5d9834fc6aa51e85c331745f4/users/f3199319c15c4ef3bdad507d18726bb9/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:18.319717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7ff6c-840b-465d-a95b-d6640a7b0efa None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:18.320030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7ff6c-840b-465d-a95b-d6640a7b0efa None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:18.320315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7ff6c-840b-465d-a95b-d6640a7b0efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:18.382906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-59e7ff6c-840b-465d-a95b-d6640a7b0efa None None] MFA Rules not processed for user `f3199319c15c4ef3bdad507d18726bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:18.384585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-59e7ff6c-840b-465d-a95b-d6640a7b0efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:18.431252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:18.570337 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 914/1841] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:18.582460 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed91e8-72d4-44ed-b049-c2f82ee813e2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:18.584741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-20ed91e8-72d4-44ed-b049-c2f82ee813e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:18.696879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef702a4-3475-48b5-a617-d666f773018b None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:18.699040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef702a4-3475-48b5-a617-d666f773018b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:18.700046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef702a4-3475-48b5-a617-d666f773018b None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:18.703998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 915/1842] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 18:00:18 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 18:00:18.741806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed91e8-72d4-44ed-b049-c2f82ee813e2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:18.748391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680f74d-347a-40c3-9a18-58578236d88b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:18.749955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680f74d-347a-40c3-9a18-58578236d88b service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:18.750144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d521a-2322-46dc-95b6-82e911185b5f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:18.750515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680f74d-347a-40c3-9a18-58578236d88b service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:18.750730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d521a-2322-46dc-95b6-82e911185b5f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:18.751315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680f74d-347a-40c3-9a18-58578236d88b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:18.751820 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d521a-2322-46dc-95b6-82e911185b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:18.773623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680f74d-347a-40c3-9a18-58578236d88b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:18.776630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680f74d-347a-40c3-9a18-58578236d88b service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:18.786270 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-dc4d521a-2322-46dc-95b6-82e911185b5f None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:18.787605 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-dc4d521a-2322-46dc-95b6-82e911185b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:18.799162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 928/1843] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:18.860446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680f74d-347a-40c3-9a18-58578236d88b service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:18.980417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 916/1844] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 18:00:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:18.985261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c912c30-a0cb-45c7-aa03-9367655e7733 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:18.986216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c912c30-a0cb-45c7-aa03-9367655e7733 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:18.986551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c912c30-a0cb-45c7-aa03-9367655e7733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:19.035001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680f74d-347a-40c3-9a18-58578236d88b service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:19.040396 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:19.041430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:19.041895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:19.042242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] PATH_INFO: `/v3/users/0f60a274b1c14cef973be40b02126463` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:19.043716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f60a274b1c14cef973be40b02126463)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:19.046075 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:19.075076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-7c912c30-a0cb-45c7-aa03-9367655e7733 None None] MFA Rules not processed for user `4408a4b02d2a4f56ae17f1ec5c23bd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:19.080472 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7c912c30-a0cb-45c7-aa03-9367655e7733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:19.235112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 917/1845] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:19.245324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c912c30-a0cb-45c7-aa03-9367655e7733 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:19.248040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7c912c30-a0cb-45c7-aa03-9367655e7733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:19.268603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c912c30-a0cb-45c7-aa03-9367655e7733 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:19.275353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:19.277057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:19.277500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:19.277846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] PATH_INFO: `/v3/projects/43345e84926347b0b1f61592243ada62` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:19.280861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=43345e84926347b0b1f61592243ada62)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:19.283636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:19.377372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f60a274b1c14cef973be40b02126463'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:19.390509 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f60a274b1c14cef973be40b02126463'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:19.392404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f60a274b1c14cef973be40b02126463'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:19.484070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f60a274b1c14cef973be40b02126463'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:19.508667 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43345e84926347b0b1f61592243ada62', 'request_id': 'req-086323cc-4257-4017-9d45-9462c5a71bfc'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:19.576134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f60a274b1c14cef973be40b02126463'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:19.587817 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 929/1846] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:18 2022] DELETE /identity/v3/users/0f60a274b1c14cef973be40b02126463 => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:19.594434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:19.611326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:19.657366 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43345e84926347b0b1f61592243ada62', 'request_id': 'req-086323cc-4257-4017-9d45-9462c5a71bfc'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:19.681764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43345e84926347b0b1f61592243ada62', 'request_id': 'req-086323cc-4257-4017-9d45-9462c5a71bfc'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:19.723078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] The token cache is being invalidate because project 43345e84926347b0b1f61592243ada62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:19.723497 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:19.725363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:19.728946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 918/1847] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:19 2022] DELETE /identity/v3/projects/43345e84926347b0b1f61592243ada62 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:19.738074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:19.738074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:19.893283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c822928-f80b-466a-b8a4-6b4d13c7d4ef admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:19.901899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a26290-19a5-43f1-bee9-c4163f68a1c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:19.902986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26290-19a5-43f1-bee9-c4163f68a1c1 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:19.903494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26290-19a5-43f1-bee9-c4163f68a1c1 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:19.903970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26290-19a5-43f1-bee9-c4163f68a1c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:19.906391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-62a26290-19a5-43f1-bee9-c4163f68a1c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:19.929139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086323cc-4257-4017-9d45-9462c5a71bfc admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:19.933411 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a26290-19a5-43f1-bee9-c4163f68a1c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:19.937864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a26290-19a5-43f1-bee9-c4163f68a1c1 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:19.970976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 930/1848] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:19.974303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e06f2d-c3db-4984-a645-f3b94af70db5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:19.974600 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e06f2d-c3db-4984-a645-f3b94af70db5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:19.976037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e06f2d-c3db-4984-a645-f3b94af70db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:19.990638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:20.028121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-00e06f2d-c3db-4984-a645-f3b94af70db5 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:20.030482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-00e06f2d-c3db-4984-a645-f3b94af70db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:20.113094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76011e20-38e6-44f1-ac9a-0384caa65454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:20.114228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76011e20-38e6-44f1-ac9a-0384caa65454 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:20.114567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76011e20-38e6-44f1-ac9a-0384caa65454 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:20.114875 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76011e20-38e6-44f1-ac9a-0384caa65454 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:20.116115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76011e20-38e6-44f1-ac9a-0384caa65454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:20.118785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76011e20-38e6-44f1-ac9a-0384caa65454 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:20.132745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 931/1849] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:20.133689 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e06f2d-c3db-4984-a645-f3b94af70db5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:20.136817 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-00e06f2d-c3db-4984-a645-f3b94af70db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:20.159180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e06f2d-c3db-4984-a645-f3b94af70db5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:20.164815 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:20.165742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:20.166024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:20.166364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] PATH_INFO: `/v3/users/9649e47995b644bca6ca9fad5d836441` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:20.167783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9649e47995b644bca6ca9fad5d836441)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:20.169728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:20.222453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 919/1850] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:19 2022] POST /identity/v3/projects => generated 374 bytes in 489 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:20.222453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76011e20-38e6-44f1-ac9a-0384caa65454 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:20.223750 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-76011e20-38e6-44f1-ac9a-0384caa65454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:20.298770 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76011e20-38e6-44f1-ac9a-0384caa65454 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:20.302969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2c9b9-683d-4169-9f9b-7a89470da903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:20.303653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2c9b9-683d-4169-9f9b-7a89470da903 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:20.303790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2c9b9-683d-4169-9f9b-7a89470da903 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:20.303958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2c9b9-683d-4169-9f9b-7a89470da903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:20.371918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9649e47995b644bca6ca9fad5d836441'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:20.383723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9649e47995b644bca6ca9fad5d836441'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:20.385175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9649e47995b644bca6ca9fad5d836441'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:20.388566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2c9b9-683d-4169-9f9b-7a89470da903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:20.390138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2c9b9-683d-4169-9f9b-7a89470da903 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:20.404762 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 920/1851] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:20.405593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2c9b9-683d-4169-9f9b-7a89470da903 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:20.415766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9649e47995b644bca6ca9fad5d836441'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:20.416347 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2c9b9-683d-4169-9f9b-7a89470da903 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:20.425528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6a600-0e29-4fc7-804c-fc9936ac2b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:20.426298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6a600-0e29-4fc7-804c-fc9936ac2b47 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:20.426651 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6a600-0e29-4fc7-804c-fc9936ac2b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:20.427055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6a600-0e29-4fc7-804c-fc9936ac2b47 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:20.427898 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6a600-0e29-4fc7-804c-fc9936ac2b47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:20.429940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6a600-0e29-4fc7-804c-fc9936ac2b47 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:20.534710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9649e47995b644bca6ca9fad5d836441'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:20.541752 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 932/1852] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:20 2022] DELETE /identity/v3/users/9649e47995b644bca6ca9fad5d836441 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:20.615648 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 921/1853] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:20 2022] POST /identity/v3/users => generated 294 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:20.619909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:20.621535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:20.700596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b43f36-262b-475a-a692-e55780a5b000 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:20.706733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffd3ca-a6bb-4201-ba1e-77fcefcc1158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:20.708120 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffd3ca-a6bb-4201-ba1e-77fcefcc1158 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:20.708120 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffd3ca-a6bb-4201-ba1e-77fcefcc1158 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:20.708120 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffd3ca-a6bb-4201-ba1e-77fcefcc1158 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:20.708811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffd3ca-a6bb-4201-ba1e-77fcefcc1158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:20.710431 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffd3ca-a6bb-4201-ba1e-77fcefcc1158 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:20.725474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 933/1854] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:20 2022] GET /identity/v3/roles => generated 1427 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:20.729679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6a600-0e29-4fc7-804c-fc9936ac2b47 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:20.749701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6a600-0e29-4fc7-804c-fc9936ac2b47 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:20.754996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dd5a3-52d3-493b-bdbd-7249ddfd68bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:20.755926 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dd5a3-52d3-493b-bdbd-7249ddfd68bd admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:20.756207 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dd5a3-52d3-493b-bdbd-7249ddfd68bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:20.756487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dd5a3-52d3-493b-bdbd-7249ddfd68bd admin admin] PATH_INFO: `/v3/projects/98b4d4734e1f47889e09ec2e8e2d96d2/users/ecad0ef54fa34683a00b8dcca96e4f08/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:20.773275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dd5a3-52d3-493b-bdbd-7249ddfd68bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b4d4734e1f47889e09ec2e8e2d96d2, user_id=ecad0ef54fa34683a00b8dcca96e4f08, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:20.776206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dd5a3-52d3-493b-bdbd-7249ddfd68bd admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:20.844461 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 922/1855] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:20 2022] PUT /identity/v3/projects/98b4d4734e1f47889e09ec2e8e2d96d2/users/ecad0ef54fa34683a00b8dcca96e4f08/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:20.855162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffd3ca-a6bb-4201-ba1e-77fcefcc1158 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:20.871188 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffd3ca-a6bb-4201-ba1e-77fcefcc1158 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:20.960995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d200fbf-2450-49d4-8085-3540f9e164a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:20.962288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d200fbf-2450-49d4-8085-3540f9e164a9 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:20.962288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d200fbf-2450-49d4-8085-3540f9e164a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:20.962584 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d200fbf-2450-49d4-8085-3540f9e164a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:20.964268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d200fbf-2450-49d4-8085-3540f9e164a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:20.965828 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d200fbf-2450-49d4-8085-3540f9e164a9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:20.979619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 934/1856] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:20 2022] GET /identity/v3/roles => generated 1427 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:20.984104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dd5a3-52d3-493b-bdbd-7249ddfd68bd admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:20.998177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dd5a3-52d3-493b-bdbd-7249ddfd68bd admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:21.007129 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f61b1a-8e95-445d-8246-f44a0487f05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:21.007129 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f61b1a-8e95-445d-8246-f44a0487f05e admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:21.007129 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f61b1a-8e95-445d-8246-f44a0487f05e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:21.007129 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f61b1a-8e95-445d-8246-f44a0487f05e admin admin] PATH_INFO: `/v3/projects/98b4d4734e1f47889e09ec2e8e2d96d2/users/ecad0ef54fa34683a00b8dcca96e4f08/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:21.008117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f61b1a-8e95-445d-8246-f44a0487f05e admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b4d4734e1f47889e09ec2e8e2d96d2, user_id=ecad0ef54fa34683a00b8dcca96e4f08, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:21.010936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f61b1a-8e95-445d-8246-f44a0487f05e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:21.024849 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 923/1857] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:20 2022] PUT /identity/v3/projects/98b4d4734e1f47889e09ec2e8e2d96d2/users/ecad0ef54fa34683a00b8dcca96e4f08/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:21.152851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d200fbf-2450-49d4-8085-3540f9e164a9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:21.155393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7d200fbf-2450-49d4-8085-3540f9e164a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:21.171687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d200fbf-2450-49d4-8085-3540f9e164a9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:21.207977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f61b1a-8e95-445d-8246-f44a0487f05e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:21.209776 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b7f61b1a-8e95-445d-8246-f44a0487f05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:21.244998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:21.246374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:21.246688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:21.247003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] PATH_INFO: `/v3/projects/0964236d544a4144b7512a8867810dc1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:21.248356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0964236d544a4144b7512a8867810dc1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:21.250913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:21.326061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f61b1a-8e95-445d-8246-f44a0487f05e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:21.337637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:21.339003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:21.339573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:21.339727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] PATH_INFO: `/v3/projects/e52436b2131246b7bc2bcd0800ac0b7a` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:21.342204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e52436b2131246b7bc2bcd0800ac0b7a)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:21.344261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:21.398469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0964236d544a4144b7512a8867810dc1', 'request_id': 'req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.469492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e52436b2131246b7bc2bcd0800ac0b7a', 'request_id': 'req-1acf95c1-35f7-4b07-872b-ed79e8544b3e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.480260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0964236d544a4144b7512a8867810dc1', 'request_id': 'req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.487538 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0964236d544a4144b7512a8867810dc1', 'request_id': 'req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.499195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] The token cache is being invalidate because project 0964236d544a4144b7512a8867810dc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:21.499654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.500447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.503690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 935/1858] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:21 2022] DELETE /identity/v3/projects/0964236d544a4144b7512a8867810dc1 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:21.505084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:21.506835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:21.558441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e52436b2131246b7bc2bcd0800ac0b7a', 'request_id': 'req-1acf95c1-35f7-4b07-872b-ed79e8544b3e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.568870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e52436b2131246b7bc2bcd0800ac0b7a', 'request_id': 'req-1acf95c1-35f7-4b07-872b-ed79e8544b3e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.591185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] The token cache is being invalidate because project e52436b2131246b7bc2bcd0800ac0b7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:21.591451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.592180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.594155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 924/1859] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:21 2022] DELETE /identity/v3/projects/e52436b2131246b7bc2bcd0800ac0b7a => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:21.595882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:21.599650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:21.621783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3fad2-0250-4c4c-bcc8-f5612d0ea880 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:21.718397 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf95c1-35f7-4b07-872b-ed79e8544b3e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:21.724667 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375d4aa-1b68-4149-9e33-38ee2a0dcbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:21.725766 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375d4aa-1b68-4149-9e33-38ee2a0dcbca admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:21.726049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375d4aa-1b68-4149-9e33-38ee2a0dcbca admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:21.726485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375d4aa-1b68-4149-9e33-38ee2a0dcbca admin admin] PATH_INFO: `/v3/users/f3199319c15c4ef3bdad507d18726bb9` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:21.728282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375d4aa-1b68-4149-9e33-38ee2a0dcbca admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3199319c15c4ef3bdad507d18726bb9)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:21.731456 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375d4aa-1b68-4149-9e33-38ee2a0dcbca admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:21.733643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d69b20-ab44-4b36-a5e4-61c4474db3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:21.734794 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d69b20-ab44-4b36-a5e4-61c4474db3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:21.735556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d69b20-ab44-4b36-a5e4-61c4474db3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:21.736057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d69b20-ab44-4b36-a5e4-61c4474db3ad admin admin] PATH_INFO: `/v3/users/762e848fc3aa41a6a94e4bec2e908866` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:21.737852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d69b20-ab44-4b36-a5e4-61c4474db3ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=762e848fc3aa41a6a94e4bec2e908866)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:21.742389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d69b20-ab44-4b36-a5e4-61c4474db3ad admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:21.953282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5375d4aa-1b68-4149-9e33-38ee2a0dcbca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3199319c15c4ef3bdad507d18726bb9'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.960864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5375d4aa-1b68-4149-9e33-38ee2a0dcbca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3199319c15c4ef3bdad507d18726bb9'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.961810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5375d4aa-1b68-4149-9e33-38ee2a0dcbca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3199319c15c4ef3bdad507d18726bb9'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:21.976163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5375d4aa-1b68-4149-9e33-38ee2a0dcbca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3199319c15c4ef3bdad507d18726bb9'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:22.018862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-64d69b20-ab44-4b36-a5e4-61c4474db3ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762e848fc3aa41a6a94e4bec2e908866'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:22.027641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-64d69b20-ab44-4b36-a5e4-61c4474db3ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762e848fc3aa41a6a94e4bec2e908866'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:22.028352 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-64d69b20-ab44-4b36-a5e4-61c4474db3ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762e848fc3aa41a6a94e4bec2e908866'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:22.041729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-64d69b20-ab44-4b36-a5e4-61c4474db3ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762e848fc3aa41a6a94e4bec2e908866'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:22.054418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5375d4aa-1b68-4149-9e33-38ee2a0dcbca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3199319c15c4ef3bdad507d18726bb9'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:22.061030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 925/1860] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:21 2022] DELETE /identity/v3/users/f3199319c15c4ef3bdad507d18726bb9 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:22.065631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dbddf-52b8-4555-ab6e-d9cccd781eff None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:22.065919 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dbddf-52b8-4555-ab6e-d9cccd781eff None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:22.066200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dbddf-52b8-4555-ab6e-d9cccd781eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:22.098031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-892dbddf-52b8-4555-ab6e-d9cccd781eff None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:22.099415 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-892dbddf-52b8-4555-ab6e-d9cccd781eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:22.133202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-64d69b20-ab44-4b36-a5e4-61c4474db3ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762e848fc3aa41a6a94e4bec2e908866'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:22.168341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 936/1861] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:21 2022] DELETE /identity/v3/users/762e848fc3aa41a6a94e4bec2e908866 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:22.172827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002fa33-f1dd-474a-8fab-3c11db3bfbf1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:22.174772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002fa33-f1dd-474a-8fab-3c11db3bfbf1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:22.175573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002fa33-f1dd-474a-8fab-3c11db3bfbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:22.211843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9002fa33-f1dd-474a-8fab-3c11db3bfbf1 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:22.215351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9002fa33-f1dd-474a-8fab-3c11db3bfbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:22.299731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 926/1862] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:22.306565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 937/1863] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:22.319316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd88a0-049c-4955-933d-f30be8ee6f6d None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:22.319644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd88a0-049c-4955-933d-f30be8ee6f6d None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:22.319929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd88a0-049c-4955-933d-f30be8ee6f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:22.341280 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:22.342450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-8edd88a0-049c-4955-933d-f30be8ee6f6d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:22.344088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002fa33-f1dd-474a-8fab-3c11db3bfbf1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:22.349780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-8edd88a0-049c-4955-933d-f30be8ee6f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:22.350469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9002fa33-f1dd-474a-8fab-3c11db3bfbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:22.357896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 927/1864] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:22.358964 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd88a0-049c-4955-933d-f30be8ee6f6d None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:22.369953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd88a0-049c-4955-933d-f30be8ee6f6d None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:22.386498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa3aa6-e339-4cf9-9487-089eb7ba215e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:22.387214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa3aa6-e339-4cf9-9487-089eb7ba215e admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:22.387497 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa3aa6-e339-4cf9-9487-089eb7ba215e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:22.387781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa3aa6-e339-4cf9-9487-089eb7ba215e admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:22.388536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa3aa6-e339-4cf9-9487-089eb7ba215e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:22.388906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa3aa6-e339-4cf9-9487-089eb7ba215e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:22.390687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa3aa6-e339-4cf9-9487-089eb7ba215e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:22.410100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 928/1865] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:22.429470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002fa33-f1dd-474a-8fab-3c11db3bfbf1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:22.431443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa3aa6-e339-4cf9-9487-089eb7ba215e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:22.435601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10563f47-0256-4434-b5fa-6b51c37d1010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:22.436559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10563f47-0256-4434-b5fa-6b51c37d1010 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:22.436911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10563f47-0256-4434-b5fa-6b51c37d1010 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:22.437254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10563f47-0256-4434-b5fa-6b51c37d1010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:22.458389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10563f47-0256-4434-b5fa-6b51c37d1010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:22.458389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10563f47-0256-4434-b5fa-6b51c37d1010 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:22.460582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa3aa6-e339-4cf9-9487-089eb7ba215e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:22.471850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f73ccd-5364-4f1a-955b-358baa7b0cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:22.473001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 938/1866] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:22.473997 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f73ccd-5364-4f1a-955b-358baa7b0cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:22.478245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f73ccd-5364-4f1a-955b-358baa7b0cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:22.478762 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f73ccd-5364-4f1a-955b-358baa7b0cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:22.479710 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f73ccd-5364-4f1a-955b-358baa7b0cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:22.481661 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f73ccd-5364-4f1a-955b-358baa7b0cb5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:22.638262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 929/1867] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:22 2022] POST /identity/v3/projects => generated 390 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:22.643484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10563f47-0256-4434-b5fa-6b51c37d1010 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:22.656523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10563f47-0256-4434-b5fa-6b51c37d1010 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:22.742540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ade17-8889-4576-915c-b431fbc4c1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:22.743313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ade17-8889-4576-915c-b431fbc4c1ef admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:22.743669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ade17-8889-4576-915c-b431fbc4c1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:22.744902 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ade17-8889-4576-915c-b431fbc4c1ef admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:22.744902 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ade17-8889-4576-915c-b431fbc4c1ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:22.746745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ade17-8889-4576-915c-b431fbc4c1ef admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:22.856249 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 939/1868] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:22 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:22.861542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f73ccd-5364-4f1a-955b-358baa7b0cb5 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:22.873749 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f73ccd-5364-4f1a-955b-358baa7b0cb5 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:22.878727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3cd539-12bb-4185-82fc-784dd59afc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:22.879497 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cd539-12bb-4185-82fc-784dd59afc64 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:22.879850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cd539-12bb-4185-82fc-784dd59afc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:22.880182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cd539-12bb-4185-82fc-784dd59afc64 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:22.881003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3cd539-12bb-4185-82fc-784dd59afc64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:22.882675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3cd539-12bb-4185-82fc-784dd59afc64 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:22.911571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 930/1869] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:22 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:22.913270 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ade17-8889-4576-915c-b431fbc4c1ef admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:22.932782 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ade17-8889-4576-915c-b431fbc4c1ef admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:22.941570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f9596-254d-4ccc-98c6-9bf4f40abb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:22.942349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f9596-254d-4ccc-98c6-9bf4f40abb39 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:22.942943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f9596-254d-4ccc-98c6-9bf4f40abb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:22.943291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f9596-254d-4ccc-98c6-9bf4f40abb39 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:22.944113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f9596-254d-4ccc-98c6-9bf4f40abb39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:22.945932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f9596-254d-4ccc-98c6-9bf4f40abb39 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:22.983685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3cd539-12bb-4185-82fc-784dd59afc64 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:22.992902 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 940/1870] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:22 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:23.008153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3cd539-12bb-4185-82fc-784dd59afc64 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:23.022001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff752df-7fa9-486e-802e-1ddc93de9af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:23.029160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff752df-7fa9-486e-802e-1ddc93de9af2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:23.030107 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff752df-7fa9-486e-802e-1ddc93de9af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:23.030452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff752df-7fa9-486e-802e-1ddc93de9af2 admin admin] PATH_INFO: `/v3/projects/38efcaf5b2ef4fb0af373574acb931df/users/bac536a8b1be4f7ba4960c86d3965929/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:23.086624 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff752df-7fa9-486e-802e-1ddc93de9af2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38efcaf5b2ef4fb0af373574acb931df, user_id=bac536a8b1be4f7ba4960c86d3965929, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:23.097220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff752df-7fa9-486e-802e-1ddc93de9af2 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:23.195936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 931/1871] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:22 2022] PUT /identity/v3/projects/38efcaf5b2ef4fb0af373574acb931df/users/bac536a8b1be4f7ba4960c86d3965929/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:23.206640 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f9596-254d-4ccc-98c6-9bf4f40abb39 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:23.207680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f413e-713c-441e-8971-377ac575bf6b None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:23.208081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f413e-713c-441e-8971-377ac575bf6b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:23.208933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f413e-713c-441e-8971-377ac575bf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:23.209570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-243f9596-254d-4ccc-98c6-9bf4f40abb39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:23.233326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f9596-254d-4ccc-98c6-9bf4f40abb39 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:23.279875 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-387f413e-713c-441e-8971-377ac575bf6b None None] MFA Rules not processed for user `bac536a8b1be4f7ba4960c86d3965929`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:23.281340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-387f413e-713c-441e-8971-377ac575bf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:23.333749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:23.334724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:23.335209 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:23.335487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] PATH_INFO: `/v3/projects/536360f5d9834fc6aa51e85c331745f4` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:23.337006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=536360f5d9834fc6aa51e85c331745f4)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:23.339191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:23.429774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 932/1872] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:23.431959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046be5b-066b-42f9-8fca-bc70d18c6306 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:23.433086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046be5b-066b-42f9-8fca-bc70d18c6306 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:23.433086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046be5b-066b-42f9-8fca-bc70d18c6306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:23.466735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-6046be5b-066b-42f9-8fca-bc70d18c6306 None None] MFA Rules not processed for user `ecad0ef54fa34683a00b8dcca96e4f08`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:23.469306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6046be5b-066b-42f9-8fca-bc70d18c6306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:23.519353 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '536360f5d9834fc6aa51e85c331745f4', 'request_id': 'req-302d9c2c-d248-424c-a437-ccd0d0837c92'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:23.593402 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 933/1873] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:23.593402 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046be5b-066b-42f9-8fca-bc70d18c6306 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:23.615338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '536360f5d9834fc6aa51e85c331745f4', 'request_id': 'req-302d9c2c-d248-424c-a437-ccd0d0837c92'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:23.625744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '536360f5d9834fc6aa51e85c331745f4', 'request_id': 'req-302d9c2c-d248-424c-a437-ccd0d0837c92'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:23.629470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046be5b-066b-42f9-8fca-bc70d18c6306 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:23.658118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] The token cache is being invalidate because project 536360f5d9834fc6aa51e85c331745f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:23.658118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:23.658118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:23.658118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 941/1874] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:23 2022] DELETE /identity/v3/projects/536360f5d9834fc6aa51e85c331745f4 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:23.663657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:23.666508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:23.752001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b68fa3-1008-4e27-9c4f-37020e8252d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:23.752974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b68fa3-1008-4e27-9c4f-37020e8252d6 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:23.753251 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b68fa3-1008-4e27-9c4f-37020e8252d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:23.753525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b68fa3-1008-4e27-9c4f-37020e8252d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:23.754229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b68fa3-1008-4e27-9c4f-37020e8252d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:23.762040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b68fa3-1008-4e27-9c4f-37020e8252d6 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:23.860434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 934/1875] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:23 2022] POST /identity/v3/projects => generated 388 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:23.860434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b68fa3-1008-4e27-9c4f-37020e8252d6 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:23.863287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f9b68fa3-1008-4e27-9c4f-37020e8252d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:23.875938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d9c2c-d248-424c-a437-ccd0d0837c92 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:23.984833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:23.992212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b68fa3-1008-4e27-9c4f-37020e8252d6 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:23.996821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0dc30-b5e5-4299-a402-76d1891e9927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:23.998430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0dc30-b5e5-4299-a402-76d1891e9927 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:23.999077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0dc30-b5e5-4299-a402-76d1891e9927 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:23.999577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0dc30-b5e5-4299-a402-76d1891e9927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:24.003802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-edc0dc30-b5e5-4299-a402-76d1891e9927 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:24.008786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6e304-e279-4524-8f7a-63dff702b02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:24.009862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6e304-e279-4524-8f7a-63dff702b02f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:24.010277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6e304-e279-4524-8f7a-63dff702b02f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:24.010624 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6e304-e279-4524-8f7a-63dff702b02f admin admin] PATH_INFO: `/v3/projects/8a83868422324650bcdc52ad61be2058` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:24.012178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6e304-e279-4524-8f7a-63dff702b02f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a83868422324650bcdc52ad61be2058)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:24.014567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6e304-e279-4524-8f7a-63dff702b02f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:24.116531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0dc30-b5e5-4299-a402-76d1891e9927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:24.118229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0dc30-b5e5-4299-a402-76d1891e9927 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:24.132712 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 935/1876] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:24.137268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cc532-e5ff-4d8a-8628-ddc6240816b2 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:24.143234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cc532-e5ff-4d8a-8628-ddc6240816b2 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:24.143522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cc532-e5ff-4d8a-8628-ddc6240816b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:24.171453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-9b8cc532-e5ff-4d8a-8628-ddc6240816b2 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:24.173829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9b8cc532-e5ff-4d8a-8628-ddc6240816b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:24.198327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-47c6e304-e279-4524-8f7a-63dff702b02f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a83868422324650bcdc52ad61be2058', 'request_id': 'req-47c6e304-e279-4524-8f7a-63dff702b02f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:24.277819 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-47c6e304-e279-4524-8f7a-63dff702b02f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a83868422324650bcdc52ad61be2058', 'request_id': 'req-47c6e304-e279-4524-8f7a-63dff702b02f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:24.285458 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-47c6e304-e279-4524-8f7a-63dff702b02f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a83868422324650bcdc52ad61be2058', 'request_id': 'req-47c6e304-e279-4524-8f7a-63dff702b02f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:24.286288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 936/1877] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:24.287296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cc532-e5ff-4d8a-8628-ddc6240816b2 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:24.295531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9b8cc532-e5ff-4d8a-8628-ddc6240816b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:24.298003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-47c6e304-e279-4524-8f7a-63dff702b02f admin admin] The token cache is being invalidate because project 8a83868422324650bcdc52ad61be2058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:24.298356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-47c6e304-e279-4524-8f7a-63dff702b02f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:24.299157 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-47c6e304-e279-4524-8f7a-63dff702b02f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:24.302887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 942/1878] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:23 2022] DELETE /identity/v3/projects/8a83868422324650bcdc52ad61be2058 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:24.302887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6d408-a27e-4aab-92ab-cab383c848d9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:24.302887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6d408-a27e-4aab-92ab-cab383c848d9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:24.302887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6d408-a27e-4aab-92ab-cab383c848d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:24.309910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cc532-e5ff-4d8a-8628-ddc6240816b2 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:24.316936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25bb1-a212-4cda-bb33-c3c168aa7745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:24.317604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25bb1-a212-4cda-bb33-c3c168aa7745 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:24.317892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25bb1-a212-4cda-bb33-c3c168aa7745 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:24.318223 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25bb1-a212-4cda-bb33-c3c168aa7745 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:24.319032 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d25bb1-a212-4cda-bb33-c3c168aa7745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:24.321374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d25bb1-a212-4cda-bb33-c3c168aa7745 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:24.326637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-efb6d408-a27e-4aab-92ab-cab383c848d9 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:24.327909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-efb6d408-a27e-4aab-92ab-cab383c848d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:24.340548 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 943/1879] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:24.341425 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6d408-a27e-4aab-92ab-cab383c848d9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:24.342873 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-efb6d408-a27e-4aab-92ab-cab383c848d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:24.353886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:24.362111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6d408-a27e-4aab-92ab-cab383c848d9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:24.363279 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:24.364213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:24.364935 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:24.364935 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] PATH_INFO: `/v3/users/4342f35dd86a4c2d861049482defd9ff` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:24.365946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4342f35dd86a4c2d861049482defd9ff)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:24.368226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:24.444316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 937/1880] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:24 2022] POST /identity/v3/users => generated 301 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:24.445188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25bb1-a212-4cda-bb33-c3c168aa7745 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:24.456018 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25bb1-a212-4cda-bb33-c3c168aa7745 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:24.460739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ea1d7-32c4-4402-810f-e00257c40501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:24.461399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ea1d7-32c4-4402-810f-e00257c40501 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:24.461677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ea1d7-32c4-4402-810f-e00257c40501 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:24.461948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ea1d7-32c4-4402-810f-e00257c40501 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:24.462767 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ea1d7-32c4-4402-810f-e00257c40501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:24.463019 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ea1d7-32c4-4402-810f-e00257c40501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:24.464601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ea1d7-32c4-4402-810f-e00257c40501 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:24.478395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 938/1881] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:24.479500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ea1d7-32c4-4402-810f-e00257c40501 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:24.482005 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ac8ea1d7-32c4-4402-810f-e00257c40501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:24.496085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ea1d7-32c4-4402-810f-e00257c40501 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:24.500897 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aa70d-9c7d-461f-ab5e-6f94f0ecdc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:24.502235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aa70d-9c7d-461f-ab5e-6f94f0ecdc7c admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:24.502509 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aa70d-9c7d-461f-ab5e-6f94f0ecdc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:24.506035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aa70d-9c7d-461f-ab5e-6f94f0ecdc7c admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:24.506035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09aa70d-9c7d-461f-ab5e-6f94f0ecdc7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:24.506035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09aa70d-9c7d-461f-ab5e-6f94f0ecdc7c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:24.521002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 939/1882] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:24 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:24.521621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aa70d-9c7d-461f-ab5e-6f94f0ecdc7c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:24.533406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aa70d-9c7d-461f-ab5e-6f94f0ecdc7c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:24.540201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c31a6-abd4-4df2-b514-7c7d3980e2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:24.540201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c31a6-abd4-4df2-b514-7c7d3980e2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:24.540201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c31a6-abd4-4df2-b514-7c7d3980e2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:24.540201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c31a6-abd4-4df2-b514-7c7d3980e2c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:24.540807 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c31a6-abd4-4df2-b514-7c7d3980e2c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:24.542555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c31a6-abd4-4df2-b514-7c7d3980e2c4 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:24.669616 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 940/1883] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:24 2022] POST /identity/v3/projects => generated 380 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:24.670867 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c31a6-abd4-4df2-b514-7c7d3980e2c4 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:24.684253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4342f35dd86a4c2d861049482defd9ff'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:24.693248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4342f35dd86a4c2d861049482defd9ff'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:24.693990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4342f35dd86a4c2d861049482defd9ff'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:24.708790 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4342f35dd86a4c2d861049482defd9ff'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:24.745822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c31a6-abd4-4df2-b514-7c7d3980e2c4 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:24.751100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd617de9-6f7a-4519-bf8a-d0d6d0af3b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:24.752550 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd617de9-6f7a-4519-bf8a-d0d6d0af3b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:24.752833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd617de9-6f7a-4519-bf8a-d0d6d0af3b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:24.753113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd617de9-6f7a-4519-bf8a-d0d6d0af3b51 admin admin] PATH_INFO: `/v3/projects/8a7a247175f142699f5d0b7a7e7949ea/users/a3e379c73ee6409e9f41b959814c6779/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:24.768774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd617de9-6f7a-4519-bf8a-d0d6d0af3b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7a247175f142699f5d0b7a7e7949ea, user_id=a3e379c73ee6409e9f41b959814c6779, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:24.773040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd617de9-6f7a-4519-bf8a-d0d6d0af3b51 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:24.820530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4342f35dd86a4c2d861049482defd9ff'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:24.828121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 944/1884] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:24 2022] DELETE /identity/v3/users/4342f35dd86a4c2d861049482defd9ff => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:24.829184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:24.831013 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:24.859527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 941/1885] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:24 2022] PUT /identity/v3/projects/8a7a247175f142699f5d0b7a7e7949ea/users/a3e379c73ee6409e9f41b959814c6779/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:24.864475 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd617de9-6f7a-4519-bf8a-d0d6d0af3b51 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:24.871124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-fd617de9-6f7a-4519-bf8a-d0d6d0af3b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:24.962856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947af184-c9a6-4c0a-9370-b6e1fc5332e1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:25.001596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd617de9-6f7a-4519-bf8a-d0d6d0af3b51 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:25.010176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dafd9-eec6-4496-b851-8c9d38a91d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:25.011089 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dafd9-eec6-4496-b851-8c9d38a91d9b admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:25.011376 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dafd9-eec6-4496-b851-8c9d38a91d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:25.012042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025dafd9-eec6-4496-b851-8c9d38a91d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:25.012491 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025dafd9-eec6-4496-b851-8c9d38a91d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:25.014604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025dafd9-eec6-4496-b851-8c9d38a91d9b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:25.028857 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 942/1886] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:24 2022] GET /identity/v3/roles => generated 1427 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:25.034112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dafd9-eec6-4496-b851-8c9d38a91d9b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:25.050755 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1108f-bff7-4088-8d5c-4918d7f5181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:25.053111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1108f-bff7-4088-8d5c-4918d7f5181f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:25.053605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1108f-bff7-4088-8d5c-4918d7f5181f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:25.054037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1108f-bff7-4088-8d5c-4918d7f5181f admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:25.055474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1108f-bff7-4088-8d5c-4918d7f5181f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:25.059817 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1108f-bff7-4088-8d5c-4918d7f5181f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:25.116973 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025dafd9-eec6-4496-b851-8c9d38a91d9b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:25.121332 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db82a41-deca-4095-ad31-de0d87da5640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:25.122860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db82a41-deca-4095-ad31-de0d87da5640 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:25.122860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db82a41-deca-4095-ad31-de0d87da5640 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:25.123098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db82a41-deca-4095-ad31-de0d87da5640 admin admin] PATH_INFO: `/v3/projects/8a7a247175f142699f5d0b7a7e7949ea/users/a3e379c73ee6409e9f41b959814c6779/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:25.131103 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db82a41-deca-4095-ad31-de0d87da5640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7a247175f142699f5d0b7a7e7949ea, user_id=a3e379c73ee6409e9f41b959814c6779, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:25.133957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db82a41-deca-4095-ad31-de0d87da5640 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:25.148578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 943/1887] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:25 2022] PUT /identity/v3/projects/8a7a247175f142699f5d0b7a7e7949ea/users/a3e379c73ee6409e9f41b959814c6779/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:25.160041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309df0c-02df-4d39-84b9-a4d7f22f3587 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:25.160405 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309df0c-02df-4d39-84b9-a4d7f22f3587 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:25.160743 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309df0c-02df-4d39-84b9-a4d7f22f3587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:25.177618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 945/1888] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:24 2022] POST /identity/v3/users => generated 291 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:25.179076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1108f-bff7-4088-8d5c-4918d7f5181f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:25.197219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1108f-bff7-4088-8d5c-4918d7f5181f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:25.266324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3309df0c-02df-4d39-84b9-a4d7f22f3587 None None] MFA Rules not processed for user `a3e379c73ee6409e9f41b959814c6779`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:25.268957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3309df0c-02df-4d39-84b9-a4d7f22f3587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:25.352949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c61f1-42f9-4a0d-a634-db1c443d5198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:25.353891 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c61f1-42f9-4a0d-a634-db1c443d5198 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:25.354179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c61f1-42f9-4a0d-a634-db1c443d5198 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:25.359502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c61f1-42f9-4a0d-a634-db1c443d5198 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:25.359502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c61f1-42f9-4a0d-a634-db1c443d5198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:25.359502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c61f1-42f9-4a0d-a634-db1c443d5198 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:25.379793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 946/1889] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:25 2022] GET /identity/v3/roles => generated 1427 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:25.384935 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c61f1-42f9-4a0d-a634-db1c443d5198 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:25.409850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c61f1-42f9-4a0d-a634-db1c443d5198 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:25.433631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 944/1890] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:25.441795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4f623-2904-4555-acaf-7557089a3c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:25.442884 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309df0c-02df-4d39-84b9-a4d7f22f3587 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:25.443195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4f623-2904-4555-acaf-7557089a3c20 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:25.443491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4f623-2904-4555-acaf-7557089a3c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:25.444357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4f623-2904-4555-acaf-7557089a3c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:25.444974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3309df0c-02df-4d39-84b9-a4d7f22f3587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:25.445170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4f623-2904-4555-acaf-7557089a3c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:25.446799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4f623-2904-4555-acaf-7557089a3c20 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:25.476521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 947/1891] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:25 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:25.481365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4f623-2904-4555-acaf-7557089a3c20 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:25.507101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4f623-2904-4555-acaf-7557089a3c20 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:25.518376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f5392-9a88-49ce-8159-3da4d053da50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:25.521743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f5392-9a88-49ce-8159-3da4d053da50 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:25.522016 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f5392-9a88-49ce-8159-3da4d053da50 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:25.522294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f5392-9a88-49ce-8159-3da4d053da50 admin admin] PATH_INFO: `/v3/projects/1c0e6d5633a94767b98471703630972c/users/e56b89c86d344bc99d5860c8ac2e95ed/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:25.547783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f5392-9a88-49ce-8159-3da4d053da50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c0e6d5633a94767b98471703630972c, user_id=e56b89c86d344bc99d5860c8ac2e95ed, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:25.551601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f5392-9a88-49ce-8159-3da4d053da50 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:25.552424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309df0c-02df-4d39-84b9-a4d7f22f3587 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:25.566636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d14f5-e8d9-4767-b01d-056d61310df6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:25.566636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d14f5-e8d9-4767-b01d-056d61310df6 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:25.566636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d14f5-e8d9-4767-b01d-056d61310df6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:25.566636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d14f5-e8d9-4767-b01d-056d61310df6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:25.581323 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d14f5-e8d9-4767-b01d-056d61310df6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:25.583081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d14f5-e8d9-4767-b01d-056d61310df6 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:25.607025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 945/1892] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:25.650127 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 948/1893] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:25 2022] PUT /identity/v3/projects/1c0e6d5633a94767b98471703630972c/users/e56b89c86d344bc99d5860c8ac2e95ed/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:25.668572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562fc50-0f5f-4314-a518-4eb0d0667a3f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:25.668853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562fc50-0f5f-4314-a518-4eb0d0667a3f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:25.669129 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562fc50-0f5f-4314-a518-4eb0d0667a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:25.716279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3562fc50-0f5f-4314-a518-4eb0d0667a3f None None] MFA Rules not processed for user `e56b89c86d344bc99d5860c8ac2e95ed`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:25.718530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3562fc50-0f5f-4314-a518-4eb0d0667a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:25.799543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f5392-9a88-49ce-8159-3da4d053da50 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:25.837626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 946/1894] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:25.842175 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562fc50-0f5f-4314-a518-4eb0d0667a3f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:25.887012 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f5392-9a88-49ce-8159-3da4d053da50 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:25.906766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:25.908647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:25.908967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:25.909252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] PATH_INFO: `/v3/users/bac536a8b1be4f7ba4960c86d3965929` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:25.911304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bac536a8b1be4f7ba4960c86d3965929)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:25.915309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:25.942789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562fc50-0f5f-4314-a518-4eb0d0667a3f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:25.951095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2a8e6-6348-4196-ab23-15ad2b698f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:25.951939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2a8e6-6348-4196-ab23-15ad2b698f8f admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:25.952223 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2a8e6-6348-4196-ab23-15ad2b698f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:25.952510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2a8e6-6348-4196-ab23-15ad2b698f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:25.954219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2a8e6-6348-4196-ab23-15ad2b698f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:25.957691 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2a8e6-6348-4196-ab23-15ad2b698f8f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:26.136039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 947/1895] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:25 2022] POST /identity/v3/projects => generated 382 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:26.139477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2a8e6-6348-4196-ab23-15ad2b698f8f admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:26.142653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-91e2a8e6-6348-4196-ab23-15ad2b698f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:26.241238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2a8e6-6348-4196-ab23-15ad2b698f8f admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:26.246976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a754606-df56-4b52-b90a-7d3f2b770537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:26.247946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a754606-df56-4b52-b90a-7d3f2b770537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:26.248248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bac536a8b1be4f7ba4960c86d3965929'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:26.248349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a754606-df56-4b52-b90a-7d3f2b770537 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:26.248654 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a754606-df56-4b52-b90a-7d3f2b770537 admin admin] PATH_INFO: `/v3/projects/8616007633fb41bba2596844fa7e6e1f` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:26.249994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a754606-df56-4b52-b90a-7d3f2b770537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8616007633fb41bba2596844fa7e6e1f)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:26.252121 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a754606-df56-4b52-b90a-7d3f2b770537 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:26.258021 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bac536a8b1be4f7ba4960c86d3965929'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:26.258908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bac536a8b1be4f7ba4960c86d3965929'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:26.277526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bac536a8b1be4f7ba4960c86d3965929'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:26.397542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bac536a8b1be4f7ba4960c86d3965929'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:26.405059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 949/1896] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:25 2022] DELETE /identity/v3/users/bac536a8b1be4f7ba4960c86d3965929 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:26.406003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:26.412101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:26.516713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e361b9c-e227-4fa0-804d-06c87da099bd admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:26.522036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e74da-209c-45c3-a956-704106248fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:26.522975 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e74da-209c-45c3-a956-704106248fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:26.523261 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e74da-209c-45c3-a956-704106248fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:26.523540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e74da-209c-45c3-a956-704106248fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:26.524309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e74da-209c-45c3-a956-704106248fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:26.526495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e74da-209c-45c3-a956-704106248fd8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:26.621161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-8a754606-df56-4b52-b90a-7d3f2b770537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8616007633fb41bba2596844fa7e6e1f', 'request_id': 'req-8a754606-df56-4b52-b90a-7d3f2b770537'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:26.634002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 950/1897] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:26 2022] POST /identity/v3/users => generated 298 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:26.638600 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e74da-209c-45c3-a956-704106248fd8 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:26.648466 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ca6e74da-209c-45c3-a956-704106248fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:26.655086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:26.742215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e74da-209c-45c3-a956-704106248fd8 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:26.747457 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-8a754606-df56-4b52-b90a-7d3f2b770537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8616007633fb41bba2596844fa7e6e1f', 'request_id': 'req-8a754606-df56-4b52-b90a-7d3f2b770537'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:26.761895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-8a754606-df56-4b52-b90a-7d3f2b770537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8616007633fb41bba2596844fa7e6e1f', 'request_id': 'req-8a754606-df56-4b52-b90a-7d3f2b770537'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:26.780992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-8a754606-df56-4b52-b90a-7d3f2b770537 admin admin] The token cache is being invalidate because project 8616007633fb41bba2596844fa7e6e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:26.781311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-8a754606-df56-4b52-b90a-7d3f2b770537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:26.787593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-8a754606-df56-4b52-b90a-7d3f2b770537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:26.792407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 948/1898] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:26 2022] DELETE /identity/v3/projects/8616007633fb41bba2596844fa7e6e1f => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:26.796095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3bba9-d743-442e-9c40-92361e050739 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:26.797848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3bba9-d743-442e-9c40-92361e050739 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:26.798641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3bba9-d743-442e-9c40-92361e050739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:26.819645 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b77a94-ebf9-4c5d-bc65-948f4a64538f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:26.820574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b77a94-ebf9-4c5d-bc65-948f4a64538f service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:26.821086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b77a94-ebf9-4c5d-bc65-948f4a64538f service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:26.821471 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b77a94-ebf9-4c5d-bc65-948f4a64538f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:26.824168 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-32b77a94-ebf9-4c5d-bc65-948f4a64538f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:26.834892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-fff3bba9-d743-442e-9c40-92361e050739 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:26.849075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-fff3bba9-d743-442e-9c40-92361e050739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:26.951050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b77a94-ebf9-4c5d-bc65-948f4a64538f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:26.952581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b77a94-ebf9-4c5d-bc65-948f4a64538f service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:26.964566 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 951/1899] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:26.965618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b77a94-ebf9-4c5d-bc65-948f4a64538f service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:26.966901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-32b77a94-ebf9-4c5d-bc65-948f4a64538f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:27.014189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 949/1900] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:27.015709 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00301c21-0e53-4a62-bc20-c1c64808d2ee None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:27.016015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00301c21-0e53-4a62-bc20-c1c64808d2ee None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:27.022969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00301c21-0e53-4a62-bc20-c1c64808d2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:27.053389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b77a94-ebf9-4c5d-bc65-948f4a64538f service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:27.070683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f8906-be5b-45a0-aaf5-5e7e12f725e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:27.071897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f8906-be5b-45a0-aaf5-5e7e12f725e0 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:27.072385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f8906-be5b-45a0-aaf5-5e7e12f725e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:27.072896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f8906-be5b-45a0-aaf5-5e7e12f725e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:27.073983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f8906-be5b-45a0-aaf5-5e7e12f725e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:27.076832 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f8906-be5b-45a0-aaf5-5e7e12f725e0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:27.087216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-00301c21-0e53-4a62-bc20-c1c64808d2ee None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:27.090838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-00301c21-0e53-4a62-bc20-c1c64808d2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:27.096523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 952/1901] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:26 2022] GET /identity/v3/roles => generated 1427 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:27.098560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f8906-be5b-45a0-aaf5-5e7e12f725e0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:27.100536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-aa3f8906-be5b-45a0-aaf5-5e7e12f725e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:27.103588 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:27.143616 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 950/1902] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:27.144669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00301c21-0e53-4a62-bc20-c1c64808d2ee None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:27.187105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00301c21-0e53-4a62-bc20-c1c64808d2ee None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:27.199060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378b050-5567-4cb3-bdf0-1bd79e28e948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:27.199940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378b050-5567-4cb3-bdf0-1bd79e28e948 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:27.200232 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378b050-5567-4cb3-bdf0-1bd79e28e948 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:27.200504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378b050-5567-4cb3-bdf0-1bd79e28e948 admin admin] PATH_INFO: `/v3/projects/874654f68c3944ce8ad75801c70ddcd8/users/723005fe72c64be3a96ee1c283364058/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:27.212693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f8906-be5b-45a0-aaf5-5e7e12f725e0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:27.217933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456207ce-314c-4063-b884-ae58e61dc0a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:27.219010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456207ce-314c-4063-b884-ae58e61dc0a7 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:27.220089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456207ce-314c-4063-b884-ae58e61dc0a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:27.220941 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456207ce-314c-4063-b884-ae58e61dc0a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:27.235506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456207ce-314c-4063-b884-ae58e61dc0a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:27.237876 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378b050-5567-4cb3-bdf0-1bd79e28e948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=874654f68c3944ce8ad75801c70ddcd8, user_id=723005fe72c64be3a96ee1c283364058, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:27.238006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456207ce-314c-4063-b884-ae58e61dc0a7 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:27.247780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378b050-5567-4cb3-bdf0-1bd79e28e948 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:27.253165 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 953/1903] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:27.254555 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217efb4-9054-4b00-9e03-087eb51856ca None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:27.255021 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217efb4-9054-4b00-9e03-087eb51856ca None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:27.255320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217efb4-9054-4b00-9e03-087eb51856ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:27.288584 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5217efb4-9054-4b00-9e03-087eb51856ca None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:27.289993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5217efb4-9054-4b00-9e03-087eb51856ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:27.302283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 954/1904] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:27.303623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217efb4-9054-4b00-9e03-087eb51856ca None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:27.322950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217efb4-9054-4b00-9e03-087eb51856ca None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:27.338569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1dfc1-c846-4a94-a535-3b57fa7ac049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:27.341344 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1dfc1-c846-4a94-a535-3b57fa7ac049 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:27.341833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1dfc1-c846-4a94-a535-3b57fa7ac049 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:27.341941 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1dfc1-c846-4a94-a535-3b57fa7ac049 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:27.342651 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1dfc1-c846-4a94-a535-3b57fa7ac049 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:27.342980 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1dfc1-c846-4a94-a535-3b57fa7ac049 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:27.344608 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1dfc1-c846-4a94-a535-3b57fa7ac049 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:27.350478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 951/1905] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:27 2022] PUT /identity/v3/projects/874654f68c3944ce8ad75801c70ddcd8/users/723005fe72c64be3a96ee1c283364058/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:27.354412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378b050-5567-4cb3-bdf0-1bd79e28e948 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:27.385409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 955/1906] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:27.394447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1dfc1-c846-4a94-a535-3b57fa7ac049 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:27.412853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1dfc1-c846-4a94-a535-3b57fa7ac049 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:27.554915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af53b2-c674-4ca8-985b-54f22b53ac23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:27.555839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af53b2-c674-4ca8-985b-54f22b53ac23 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:27.556305 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af53b2-c674-4ca8-985b-54f22b53ac23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:27.556461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af53b2-c674-4ca8-985b-54f22b53ac23 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:27.557520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af53b2-c674-4ca8-985b-54f22b53ac23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:27.559786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af53b2-c674-4ca8-985b-54f22b53ac23 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:27.562338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378b050-5567-4cb3-bdf0-1bd79e28e948 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:27.567547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcb25f-b70b-428a-b7c9-e560cdea0038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:27.568271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcb25f-b70b-428a-b7c9-e560cdea0038 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:27.568555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcb25f-b70b-428a-b7c9-e560cdea0038 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:27.568833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcb25f-b70b-428a-b7c9-e560cdea0038 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:27.569579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcb25f-b70b-428a-b7c9-e560cdea0038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:27.571244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcb25f-b70b-428a-b7c9-e560cdea0038 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:27.582509 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 952/1907] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:27 2022] GET /identity/v3/roles => generated 1427 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:27.586701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcb25f-b70b-428a-b7c9-e560cdea0038 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:27.597392 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcb25f-b70b-428a-b7c9-e560cdea0038 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:27.602103 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5496aa-b066-4795-a4b0-cb98ea89e4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:27.603071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5496aa-b066-4795-a4b0-cb98ea89e4e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:27.603344 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5496aa-b066-4795-a4b0-cb98ea89e4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:27.603627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5496aa-b066-4795-a4b0-cb98ea89e4e9 admin admin] PATH_INFO: `/v3/projects/874654f68c3944ce8ad75801c70ddcd8/users/723005fe72c64be3a96ee1c283364058/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:27.605635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5496aa-b066-4795-a4b0-cb98ea89e4e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=874654f68c3944ce8ad75801c70ddcd8, user_id=723005fe72c64be3a96ee1c283364058, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:27.608382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5496aa-b066-4795-a4b0-cb98ea89e4e9 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:27.619666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 953/1908] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:27 2022] PUT /identity/v3/projects/874654f68c3944ce8ad75801c70ddcd8/users/723005fe72c64be3a96ee1c283364058/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:27.631146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dcdf2-1546-47f2-8469-ebe360825bb3 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:27.631510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dcdf2-1546-47f2-8469-ebe360825bb3 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:27.631861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dcdf2-1546-47f2-8469-ebe360825bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:27.661961 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a94dcdf2-1546-47f2-8469-ebe360825bb3 None None] MFA Rules not processed for user `723005fe72c64be3a96ee1c283364058`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:27.663221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a94dcdf2-1546-47f2-8469-ebe360825bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:27.722277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 956/1909] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:27 2022] POST /identity/v3/projects => generated 364 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:27.726647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af53b2-c674-4ca8-985b-54f22b53ac23 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:27.732819 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 954/1910] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:27.761450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94dcdf2-1546-47f2-8469-ebe360825bb3 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:27.765843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a94dcdf2-1546-47f2-8469-ebe360825bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:27.832093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af53b2-c674-4ca8-985b-54f22b53ac23 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:27.874545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06574d-d252-4780-a951-fcc7dcf9a203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:27.875289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06574d-d252-4780-a951-fcc7dcf9a203 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:27.875551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06574d-d252-4780-a951-fcc7dcf9a203 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:27.875868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06574d-d252-4780-a951-fcc7dcf9a203 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:27.877052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06574d-d252-4780-a951-fcc7dcf9a203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:27.886065 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06574d-d252-4780-a951-fcc7dcf9a203 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:27.908596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94dcdf2-1546-47f2-8469-ebe360825bb3 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:27.987939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 957/1911] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:27 2022] POST /identity/v3/users => generated 283 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:27.988910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06574d-d252-4780-a951-fcc7dcf9a203 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:27.991351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0c06574d-d252-4780-a951-fcc7dcf9a203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:28.001377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d08ec-bbd7-4dfb-ae57-399694a3e656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:28.002344 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d08ec-bbd7-4dfb-ae57-399694a3e656 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:28.003166 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d08ec-bbd7-4dfb-ae57-399694a3e656 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:28.003460 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d08ec-bbd7-4dfb-ae57-399694a3e656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:28.006195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06574d-d252-4780-a951-fcc7dcf9a203 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:28.015516 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:28.016578 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:28.017036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:28.017362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] PATH_INFO: `/v3/projects/38efcaf5b2ef4fb0af373574acb931df` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:28.019327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38efcaf5b2ef4fb0af373574acb931df)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:28.022592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:28.084246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d08ec-bbd7-4dfb-ae57-399694a3e656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:28.085916 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d08ec-bbd7-4dfb-ae57-399694a3e656 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:28.101614 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 955/1912] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:28.107328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d08ec-bbd7-4dfb-ae57-399694a3e656 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:28.118147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d08ec-bbd7-4dfb-ae57-399694a3e656 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:28.123199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea818076-2c68-494b-b34c-f05918b585bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:28.124066 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea818076-2c68-494b-b34c-f05918b585bd admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:28.124521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea818076-2c68-494b-b34c-f05918b585bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:28.124935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea818076-2c68-494b-b34c-f05918b585bd admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:28.125529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea818076-2c68-494b-b34c-f05918b585bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:28.127196 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea818076-2c68-494b-b34c-f05918b585bd admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:28.144683 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 956/1913] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:28 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:28.148276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea818076-2c68-494b-b34c-f05918b585bd admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:28.159605 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea818076-2c68-494b-b34c-f05918b585bd admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:28.164310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f6042-b954-4d6c-b9f0-1fc4d1ea2e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:28.165127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f6042-b954-4d6c-b9f0-1fc4d1ea2e57 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:28.165361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f6042-b954-4d6c-b9f0-1fc4d1ea2e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:28.165706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f6042-b954-4d6c-b9f0-1fc4d1ea2e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:28.166366 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f6042-b954-4d6c-b9f0-1fc4d1ea2e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:28.168049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f6042-b954-4d6c-b9f0-1fc4d1ea2e57 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:28.183183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 957/1914] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:28 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:28.184967 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f6042-b954-4d6c-b9f0-1fc4d1ea2e57 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:28.200299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38efcaf5b2ef4fb0af373574acb931df', 'request_id': 'req-42f0003d-2cb6-4bcb-b058-e65c85e31444'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:28.208086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f6042-b954-4d6c-b9f0-1fc4d1ea2e57 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:28.272816 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a345e5-7f86-4219-bbad-60b62f9ff647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:28.273724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a345e5-7f86-4219-bbad-60b62f9ff647 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:28.274002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a345e5-7f86-4219-bbad-60b62f9ff647 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:28.274276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a345e5-7f86-4219-bbad-60b62f9ff647 admin admin] PATH_INFO: `/v3/projects/1482d4c5ea0941b795e12d7879e256f0/users/5a5dd73d80644aeca340746edae638eb/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:28.291703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a345e5-7f86-4219-bbad-60b62f9ff647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1482d4c5ea0941b795e12d7879e256f0, user_id=5a5dd73d80644aeca340746edae638eb, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:28.294951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a345e5-7f86-4219-bbad-60b62f9ff647 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:28.303344 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38efcaf5b2ef4fb0af373574acb931df', 'request_id': 'req-42f0003d-2cb6-4bcb-b058-e65c85e31444'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:28.311802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38efcaf5b2ef4fb0af373574acb931df', 'request_id': 'req-42f0003d-2cb6-4bcb-b058-e65c85e31444'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:28.332818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] The token cache is being invalidate because project 38efcaf5b2ef4fb0af373574acb931df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:28.337885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:28.337885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:28.337885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 958/1915] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:27 2022] DELETE /identity/v3/projects/38efcaf5b2ef4fb0af373574acb931df => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:28.340659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:28.342174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:28.355918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0003d-2cb6-4bcb-b058-e65c85e31444 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:28.362719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:28.363761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:28.364172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:28.364495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] PATH_INFO: `/v3/users/a3e379c73ee6409e9f41b959814c6779` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:28.365836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e379c73ee6409e9f41b959814c6779)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:28.367997 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:28.404502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:28.450386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 958/1916] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:28 2022] PUT /identity/v3/projects/1482d4c5ea0941b795e12d7879e256f0/users/5a5dd73d80644aeca340746edae638eb/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:28.471388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77243449-e541-43cb-99fa-03bbac86387c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:28.471388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77243449-e541-43cb-99fa-03bbac86387c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:28.471388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77243449-e541-43cb-99fa-03bbac86387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:28.496665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-77243449-e541-43cb-99fa-03bbac86387c None None] MFA Rules not processed for user `5a5dd73d80644aeca340746edae638eb`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:28.497900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-77243449-e541-43cb-99fa-03bbac86387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:28.568250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 959/1917] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:28.572918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77243449-e541-43cb-99fa-03bbac86387c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:28.574625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-77243449-e541-43cb-99fa-03bbac86387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:28.597807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e379c73ee6409e9f41b959814c6779'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:28.605839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e379c73ee6409e9f41b959814c6779'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:28.606851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e379c73ee6409e9f41b959814c6779'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:28.620803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e379c73ee6409e9f41b959814c6779'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:28.654229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77243449-e541-43cb-99fa-03bbac86387c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:28.660138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780a681-a4f4-405e-ab4e-82b116b92f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:28.661293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780a681-a4f4-405e-ab4e-82b116b92f91 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:28.661293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780a681-a4f4-405e-ab4e-82b116b92f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:28.661492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780a681-a4f4-405e-ab4e-82b116b92f91 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:28.662043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780a681-a4f4-405e-ab4e-82b116b92f91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:28.664831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780a681-a4f4-405e-ab4e-82b116b92f91 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:28.706012 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e379c73ee6409e9f41b959814c6779'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:28.713088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 959/1918] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:28 2022] DELETE /identity/v3/users/a3e379c73ee6409e9f41b959814c6779 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:28.825035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 960/1919] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:28 2022] POST /identity/v3/projects => generated 366 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:28.826259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:28.827750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:28.908663 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b001e-9852-476f-97b1-cc3e84320553 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:29.005304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed2233-549f-4ea5-8f50-2b45e84dd39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:29.005304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed2233-549f-4ea5-8f50-2b45e84dd39d admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:29.005304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed2233-549f-4ea5-8f50-2b45e84dd39d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:29.005304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed2233-549f-4ea5-8f50-2b45e84dd39d admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:29.006032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed2233-549f-4ea5-8f50-2b45e84dd39d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:29.009702 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed2233-549f-4ea5-8f50-2b45e84dd39d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:29.095189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 960/1920] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:28 2022] POST /identity/v3/users => generated 290 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:29.099206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780a681-a4f4-405e-ab4e-82b116b92f91 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:29.112226 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780a681-a4f4-405e-ab4e-82b116b92f91 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:29.117914 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e078efc-5f39-4a8a-a762-422d4b1b4255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:29.118897 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e078efc-5f39-4a8a-a762-422d4b1b4255 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:29.119292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e078efc-5f39-4a8a-a762-422d4b1b4255 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:29.119647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e078efc-5f39-4a8a-a762-422d4b1b4255 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:29.120573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e078efc-5f39-4a8a-a762-422d4b1b4255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:29.123053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e078efc-5f39-4a8a-a762-422d4b1b4255 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:29.136585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 961/1921] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:29 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:29.143319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed2233-549f-4ea5-8f50-2b45e84dd39d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:29.155336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed2233-549f-4ea5-8f50-2b45e84dd39d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:29.162142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab6da6-190f-4550-8f42-88d5fecbcf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:29.163089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab6da6-190f-4550-8f42-88d5fecbcf0c admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:29.163374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab6da6-190f-4550-8f42-88d5fecbcf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:29.166539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab6da6-190f-4550-8f42-88d5fecbcf0c admin admin] PATH_INFO: `/v3/projects/43205e3676e64b0eb4dd4d0eb382e5e7/users/45eb46fb5c314bf389ba5e7e40385452/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:29.181103 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab6da6-190f-4550-8f42-88d5fecbcf0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43205e3676e64b0eb4dd4d0eb382e5e7, user_id=45eb46fb5c314bf389ba5e7e40385452, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:29.183823 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab6da6-190f-4550-8f42-88d5fecbcf0c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:29.206802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e078efc-5f39-4a8a-a762-422d4b1b4255 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:29.207269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9e078efc-5f39-4a8a-a762-422d4b1b4255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:29.221468 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e078efc-5f39-4a8a-a762-422d4b1b4255 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:29.226571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:29.227524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:29.228004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:29.228276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] PATH_INFO: `/v3/projects/8a7a247175f142699f5d0b7a7e7949ea` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:29.229862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7a247175f142699f5d0b7a7e7949ea)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:29.231608 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:29.319188 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 961/1922] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:29 2022] PUT /identity/v3/projects/43205e3676e64b0eb4dd4d0eb382e5e7/users/45eb46fb5c314bf389ba5e7e40385452/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:29.323351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab6da6-190f-4550-8f42-88d5fecbcf0c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:29.406174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab6da6-190f-4550-8f42-88d5fecbcf0c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:29.412484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd807016-a5c2-42c5-b21f-8b2dfa14c72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:29.413215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd807016-a5c2-42c5-b21f-8b2dfa14c72a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:29.416654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd807016-a5c2-42c5-b21f-8b2dfa14c72a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:29.416654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd807016-a5c2-42c5-b21f-8b2dfa14c72a admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:29.416654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd807016-a5c2-42c5-b21f-8b2dfa14c72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:29.416902 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd807016-a5c2-42c5-b21f-8b2dfa14c72a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:29.428635 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 962/1923] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:29 2022] GET /identity/v3/roles => generated 1427 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:29.429454 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd807016-a5c2-42c5-b21f-8b2dfa14c72a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:29.431256 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-dd807016-a5c2-42c5-b21f-8b2dfa14c72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:29.432787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7a247175f142699f5d0b7a7e7949ea', 'request_id': 'req-2bba87e8-d674-4a3e-97ce-fd7f480a226f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:29.513849 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd807016-a5c2-42c5-b21f-8b2dfa14c72a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:29.515301 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7a247175f142699f5d0b7a7e7949ea', 'request_id': 'req-2bba87e8-d674-4a3e-97ce-fd7f480a226f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:29.524263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7a247175f142699f5d0b7a7e7949ea', 'request_id': 'req-2bba87e8-d674-4a3e-97ce-fd7f480a226f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:29.539170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] The token cache is being invalidate because project 8a7a247175f142699f5d0b7a7e7949ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:29.539440 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:29.540606 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:29.543695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:29.547692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 962/1924] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:29 2022] DELETE /identity/v3/projects/8a7a247175f142699f5d0b7a7e7949ea => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:29.549031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:29.551672 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:29.589120 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd458fd5-767d-435e-92b2-e0b004223b45 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '723005fe72c64be3a96ee1c283364058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '874654f68c3944ce8ad75801c70ddcd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:29.589970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd458fd5-767d-435e-92b2-e0b004223b45 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:29.590241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd458fd5-767d-435e-92b2-e0b004223b45 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:29.590526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd458fd5-767d-435e-92b2-e0b004223b45 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:29.591291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd458fd5-767d-435e-92b2-e0b004223b45 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:29.591667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd458fd5-767d-435e-92b2-e0b004223b45 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:29.597994 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd458fd5-767d-435e-92b2-e0b004223b45 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:29.614724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 963/1925] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:29.616187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387094dc-9bfe-4dc2-9438-e7fb1205d52c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:29.616485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387094dc-9bfe-4dc2-9438-e7fb1205d52c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:29.616763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387094dc-9bfe-4dc2-9438-e7fb1205d52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:29.637729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-387094dc-9bfe-4dc2-9438-e7fb1205d52c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:29.639174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-387094dc-9bfe-4dc2-9438-e7fb1205d52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:29.675484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba87e8-d674-4a3e-97ce-fd7f480a226f admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:29.680923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaaec6-e0f7-4d52-8f48-78c4ec50c05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:29.682041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaaec6-e0f7-4d52-8f48-78c4ec50c05f admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:29.682323 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaaec6-e0f7-4d52-8f48-78c4ec50c05f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:29.682646 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaaec6-e0f7-4d52-8f48-78c4ec50c05f admin admin] PATH_INFO: `/v3/projects/43205e3676e64b0eb4dd4d0eb382e5e7/users/45eb46fb5c314bf389ba5e7e40385452/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:29.684839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aaaec6-e0f7-4d52-8f48-78c4ec50c05f admin admin] RBAC: Authorizing `identity:create_grant(project_id=43205e3676e64b0eb4dd4d0eb382e5e7, user_id=45eb46fb5c314bf389ba5e7e40385452, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:29.687812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aaaec6-e0f7-4d52-8f48-78c4ec50c05f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:29.710944 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 963/1926] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:29 2022] PUT /identity/v3/projects/43205e3676e64b0eb4dd4d0eb382e5e7/users/45eb46fb5c314bf389ba5e7e40385452/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:29.712213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaaec6-e0f7-4d52-8f48-78c4ec50c05f admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:29.716149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-94aaaec6-e0f7-4d52-8f48-78c4ec50c05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:29.732369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 964/1927] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:29.732369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5607b69-92ac-43da-8233-d92122351ed3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:29.732369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5607b69-92ac-43da-8233-d92122351ed3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:29.732369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5607b69-92ac-43da-8233-d92122351ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:29.776391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d5607b69-92ac-43da-8233-d92122351ed3 None None] MFA Rules not processed for user `45eb46fb5c314bf389ba5e7e40385452`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:29.778198 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d5607b69-92ac-43da-8233-d92122351ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:29.803884 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaaec6-e0f7-4d52-8f48-78c4ec50c05f admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:29.810101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05aabb-963f-4bc4-bb75-f02b4def8cb7 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '723005fe72c64be3a96ee1c283364058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '874654f68c3944ce8ad75801c70ddcd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:29.811000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05aabb-963f-4bc4-bb75-f02b4def8cb7 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:29.811319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05aabb-963f-4bc4-bb75-f02b4def8cb7 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:29.811804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05aabb-963f-4bc4-bb75-f02b4def8cb7 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:29.812762 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05aabb-963f-4bc4-bb75-f02b4def8cb7 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:29.815506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05aabb-963f-4bc4-bb75-f02b4def8cb7 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:29.864315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 965/1928] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:29.866011 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d1e49-e05e-42f1-8cbe-824c31048933 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:29.866302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d1e49-e05e-42f1-8cbe-824c31048933 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:29.866677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d1e49-e05e-42f1-8cbe-824c31048933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:29.891646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e03d1e49-e05e-42f1-8cbe-824c31048933 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:29.893438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e03d1e49-e05e-42f1-8cbe-824c31048933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:29.952902 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 964/1929] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:29 2022] POST /identity/v3/projects => generated 355 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:29.954013 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05aabb-963f-4bc4-bb75-f02b4def8cb7 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:29.957172 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ea05aabb-963f-4bc4-bb75-f02b4def8cb7 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:29.968479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 966/1930] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:29.969146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d1e49-e05e-42f1-8cbe-824c31048933 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:29.979873 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d1e49-e05e-42f1-8cbe-824c31048933 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:30.025110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05aabb-963f-4bc4-bb75-f02b4def8cb7 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:30.029094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5ae42-5e75-4624-9eb5-117be9856dca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:30.029764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5ae42-5e75-4624-9eb5-117be9856dca service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:30.030045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5ae42-5e75-4624-9eb5-117be9856dca service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:30.030314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5ae42-5e75-4624-9eb5-117be9856dca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:30.042393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dc08f-35ba-4326-b60f-c0953798e659 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '723005fe72c64be3a96ee1c283364058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '874654f68c3944ce8ad75801c70ddcd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:30.043174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dc08f-35ba-4326-b60f-c0953798e659 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:30.043462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dc08f-35ba-4326-b60f-c0953798e659 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:30.043741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dc08f-35ba-4326-b60f-c0953798e659 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:30.044498 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113dc08f-35ba-4326-b60f-c0953798e659 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:30.044812 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113dc08f-35ba-4326-b60f-c0953798e659 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:30.046409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113dc08f-35ba-4326-b60f-c0953798e659 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:30.061467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 967/1931] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:30.061467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dc08f-35ba-4326-b60f-c0953798e659 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:30.097862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f5ae42-5e75-4624-9eb5-117be9856dca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:30.099634 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f5ae42-5e75-4624-9eb5-117be9856dca service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:30.110430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 965/1932] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:30.112123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5ae42-5e75-4624-9eb5-117be9856dca service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:30.119697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:30.122940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5ae42-5e75-4624-9eb5-117be9856dca service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:30.128933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '723005fe72c64be3a96ee1c283364058', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '874654f68c3944ce8ad75801c70ddcd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:30.129907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:30.130250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:30.130595 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] PATH_INFO: `/v3/projects/0f0790142cc441d8a3469130fee572cd` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:30.131876 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0f0790142cc441d8a3469130fee572cd)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:30.133834 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:30.138116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dc08f-35ba-4326-b60f-c0953798e659 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:30.142860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979e01d-aac6-4ba9-8383-2ef63393e23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:30.143534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979e01d-aac6-4ba9-8383-2ef63393e23a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:30.144175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979e01d-aac6-4ba9-8383-2ef63393e23a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:30.144455 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979e01d-aac6-4ba9-8383-2ef63393e23a admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:30.145309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979e01d-aac6-4ba9-8383-2ef63393e23a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:30.145616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979e01d-aac6-4ba9-8383-2ef63393e23a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:30.147254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979e01d-aac6-4ba9-8383-2ef63393e23a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:30.164263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 968/1933] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:30.166215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979e01d-aac6-4ba9-8383-2ef63393e23a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:30.177593 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979e01d-aac6-4ba9-8383-2ef63393e23a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:30.182345 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16ab86-af8e-4537-9a3d-29d16a8b4f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:30.183375 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16ab86-af8e-4537-9a3d-29d16a8b4f6f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:30.183375 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16ab86-af8e-4537-9a3d-29d16a8b4f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:30.184779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16ab86-af8e-4537-9a3d-29d16a8b4f6f admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:30.184779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16ab86-af8e-4537-9a3d-29d16a8b4f6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:30.186356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16ab86-af8e-4537-9a3d-29d16a8b4f6f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:30.260656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0790142cc441d8a3469130fee572cd', 'request_id': 'req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:30.325037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 969/1934] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:30 2022] POST /identity/v3/projects => generated 408 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:30.329160 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16ab86-af8e-4537-9a3d-29d16a8b4f6f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:30.343799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0790142cc441d8a3469130fee572cd', 'request_id': 'req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:30.346239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:30.356947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0790142cc441d8a3469130fee572cd', 'request_id': 'req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:30.371047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] The token cache is being invalidate because project 0f0790142cc441d8a3469130fee572cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:30.371336 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:30.372021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:30.376117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 966/1935] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:30 2022] DELETE /identity/v3/projects/0f0790142cc441d8a3469130fee572cd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:30.376117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:30.376117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:30.397821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16ab86-af8e-4537-9a3d-29d16a8b4f6f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:30.407547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76758505-8ef2-42fa-b922-09dd1d79bba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:30.408474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76758505-8ef2-42fa-b922-09dd1d79bba1 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:30.408761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76758505-8ef2-42fa-b922-09dd1d79bba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:30.409052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76758505-8ef2-42fa-b922-09dd1d79bba1 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:30.409836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76758505-8ef2-42fa-b922-09dd1d79bba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:30.411806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76758505-8ef2-42fa-b922-09dd1d79bba1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:30.437786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ff2d8-0599-4c19-94a4-a90aedcfbfd5 tempest-VolumeQuotaClassesTest-1713466195 tempest-VolumeQuotaClassesTest-1713466195-project-admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:30.492358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf357b5-19a2-4f1d-8596-50c98ed6bd7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:30.493102 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf357b5-19a2-4f1d-8596-50c98ed6bd7c service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:30.493385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf357b5-19a2-4f1d-8596-50c98ed6bd7c service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:30.493662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf357b5-19a2-4f1d-8596-50c98ed6bd7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:30.495511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ddf357b5-19a2-4f1d-8596-50c98ed6bd7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:30.549292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 970/1936] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:30 2022] POST /identity/v3/users => generated 305 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:30.553764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76758505-8ef2-42fa-b922-09dd1d79bba1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:30.557343 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-76758505-8ef2-42fa-b922-09dd1d79bba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:30.562752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf357b5-19a2-4f1d-8596-50c98ed6bd7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:30.568688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf357b5-19a2-4f1d-8596-50c98ed6bd7c service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:30.571286 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76758505-8ef2-42fa-b922-09dd1d79bba1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:30.577176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a1608-f0d0-4ccd-b8f2-90efaae5665e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:30.577978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a1608-f0d0-4ccd-b8f2-90efaae5665e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:30.578329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a1608-f0d0-4ccd-b8f2-90efaae5665e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:30.579050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a1608-f0d0-4ccd-b8f2-90efaae5665e admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:30.579957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a1608-f0d0-4ccd-b8f2-90efaae5665e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:30.581940 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a1608-f0d0-4ccd-b8f2-90efaae5665e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:30.584734 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 967/1937] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:30.594606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 971/1938] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:30 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:30.598568 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf357b5-19a2-4f1d-8596-50c98ed6bd7c service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:30.612094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf357b5-19a2-4f1d-8596-50c98ed6bd7c service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:30.617681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a37308-9612-43a3-8ca3-ee276de1d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:30.618370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a37308-9612-43a3-8ca3-ee276de1d6ac admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:30.618857 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a37308-9612-43a3-8ca3-ee276de1d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:30.619069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a37308-9612-43a3-8ca3-ee276de1d6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:30.619978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a37308-9612-43a3-8ca3-ee276de1d6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:30.621596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a37308-9612-43a3-8ca3-ee276de1d6ac admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:30.633925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 968/1939] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:30 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:30.638217 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a1608-f0d0-4ccd-b8f2-90efaae5665e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:30.649048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a1608-f0d0-4ccd-b8f2-90efaae5665e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:30.654348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55dfc7-3fa4-4dd7-bc49-7df3ac04f43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:30.655529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55dfc7-3fa4-4dd7-bc49-7df3ac04f43f admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:30.655908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55dfc7-3fa4-4dd7-bc49-7df3ac04f43f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:30.656176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55dfc7-3fa4-4dd7-bc49-7df3ac04f43f admin admin] PATH_INFO: `/v3/projects/ad64221d33dd460b93aec895d46c8765/users/cc27d0ab5aa44fe49c1e5a717b7a4d2d/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:30.675053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55dfc7-3fa4-4dd7-bc49-7df3ac04f43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad64221d33dd460b93aec895d46c8765, user_id=cc27d0ab5aa44fe49c1e5a717b7a4d2d, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:30.677868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55dfc7-3fa4-4dd7-bc49-7df3ac04f43f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:30.790539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 972/1940] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:30 2022] PUT /identity/v3/projects/ad64221d33dd460b93aec895d46c8765/users/cc27d0ab5aa44fe49c1e5a717b7a4d2d/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:30.802000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decb449-e43c-490a-86e0-281a9bf04c45 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:30.802288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decb449-e43c-490a-86e0-281a9bf04c45 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:30.802571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decb449-e43c-490a-86e0-281a9bf04c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:30.835088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-7decb449-e43c-490a-86e0-281a9bf04c45 None None] MFA Rules not processed for user `cc27d0ab5aa44fe49c1e5a717b7a4d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:30.836349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7decb449-e43c-490a-86e0-281a9bf04c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:30.911288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 969/1941] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:30 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:30.916366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55dfc7-3fa4-4dd7-bc49-7df3ac04f43f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:31.006588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55dfc7-3fa4-4dd7-bc49-7df3ac04f43f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:31.012923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1c863-762c-46f3-863a-ee9fb04b02f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:31.013668 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1c863-762c-46f3-863a-ee9fb04b02f9 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:31.014074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1c863-762c-46f3-863a-ee9fb04b02f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:31.014453 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1c863-762c-46f3-863a-ee9fb04b02f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:31.015299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1c863-762c-46f3-863a-ee9fb04b02f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:31.017326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1c863-762c-46f3-863a-ee9fb04b02f9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:31.178680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 973/1942] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:30 2022] POST /identity/v3/projects => generated 408 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:31.181152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decb449-e43c-490a-86e0-281a9bf04c45 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:31.192189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decb449-e43c-490a-86e0-281a9bf04c45 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:31.241599 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1c863-762c-46f3-863a-ee9fb04b02f9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:31.243430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-03b1c863-762c-46f3-863a-ee9fb04b02f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:31.268930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222a3aa-e4db-4368-993c-1ccd0dbfc3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:31.269682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222a3aa-e4db-4368-993c-1ccd0dbfc3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:31.270286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222a3aa-e4db-4368-993c-1ccd0dbfc3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:31.270286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222a3aa-e4db-4368-993c-1ccd0dbfc3c7 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:31.275046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222a3aa-e4db-4368-993c-1ccd0dbfc3c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:31.276995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222a3aa-e4db-4368-993c-1ccd0dbfc3c7 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:31.330524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1c863-762c-46f3-863a-ee9fb04b02f9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:31.335440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064f7b5-d2f4-41ec-9211-e0475d516147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:31.336322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064f7b5-d2f4-41ec-9211-e0475d516147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:31.336606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064f7b5-d2f4-41ec-9211-e0475d516147 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:31.336881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064f7b5-d2f4-41ec-9211-e0475d516147 admin admin] PATH_INFO: `/v3/users/e56b89c86d344bc99d5860c8ac2e95ed` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:31.338098 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064f7b5-d2f4-41ec-9211-e0475d516147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e56b89c86d344bc99d5860c8ac2e95ed)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:31.340155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064f7b5-d2f4-41ec-9211-e0475d516147 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:31.435177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 970/1943] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:31 2022] POST /identity/v3/users => generated 311 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:31.436698 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222a3aa-e4db-4368-993c-1ccd0dbfc3c7 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:31.446959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222a3aa-e4db-4368-993c-1ccd0dbfc3c7 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:31.451445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0eb75-cacb-42d9-a1db-7b4e29ab901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:31.452093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0eb75-cacb-42d9-a1db-7b4e29ab901a admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:31.452377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0eb75-cacb-42d9-a1db-7b4e29ab901a admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:31.452653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0eb75-cacb-42d9-a1db-7b4e29ab901a admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:31.453367 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0eb75-cacb-42d9-a1db-7b4e29ab901a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:31.455207 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0eb75-cacb-42d9-a1db-7b4e29ab901a admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:31.465472 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 971/1944] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:31 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:31.469816 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0eb75-cacb-42d9-a1db-7b4e29ab901a admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:31.480702 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0eb75-cacb-42d9-a1db-7b4e29ab901a admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:31.485500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4908e76-12ad-4e6f-94c9-8bae3f1d942e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:31.487234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4908e76-12ad-4e6f-94c9-8bae3f1d942e admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:31.487545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4908e76-12ad-4e6f-94c9-8bae3f1d942e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:31.487823 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4908e76-12ad-4e6f-94c9-8bae3f1d942e admin admin] PATH_INFO: `/v3/projects/20bbd9ef0f234d3da52d2445122237b4/users/c4ed42b1681e4a5b8c1afe867701c77a/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:31.503062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4908e76-12ad-4e6f-94c9-8bae3f1d942e admin admin] RBAC: Authorizing `identity:create_grant(project_id=20bbd9ef0f234d3da52d2445122237b4, user_id=c4ed42b1681e4a5b8c1afe867701c77a, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:31.505710 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4908e76-12ad-4e6f-94c9-8bae3f1d942e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:31.609270 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 972/1945] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:31 2022] PUT /identity/v3/projects/20bbd9ef0f234d3da52d2445122237b4/users/c4ed42b1681e4a5b8c1afe867701c77a/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:31.614047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4908e76-12ad-4e6f-94c9-8bae3f1d942e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:31.638340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f064f7b5-d2f4-41ec-9211-e0475d516147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e56b89c86d344bc99d5860c8ac2e95ed'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:31.646424 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f064f7b5-d2f4-41ec-9211-e0475d516147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e56b89c86d344bc99d5860c8ac2e95ed'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:31.647166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f064f7b5-d2f4-41ec-9211-e0475d516147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e56b89c86d344bc99d5860c8ac2e95ed'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:31.662211 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f064f7b5-d2f4-41ec-9211-e0475d516147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e56b89c86d344bc99d5860c8ac2e95ed'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:31.683358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4908e76-12ad-4e6f-94c9-8bae3f1d942e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:31.688019 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fa465-0eff-45d6-aa7e-33ad520d670b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:31.688700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fa465-0eff-45d6-aa7e-33ad520d670b admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:31.689133 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fa465-0eff-45d6-aa7e-33ad520d670b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:31.689839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fa465-0eff-45d6-aa7e-33ad520d670b admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:31.690647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477fa465-0eff-45d6-aa7e-33ad520d670b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:31.692305 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477fa465-0eff-45d6-aa7e-33ad520d670b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:31.705464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 973/1946] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:31 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:31.707613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fa465-0eff-45d6-aa7e-33ad520d670b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:31.719058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fa465-0eff-45d6-aa7e-33ad520d670b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:31.725793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd22c-2e05-43d9-a898-1e5954fb11a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:31.726198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd22c-2e05-43d9-a898-1e5954fb11a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:31.726406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd22c-2e05-43d9-a898-1e5954fb11a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:31.726676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd22c-2e05-43d9-a898-1e5954fb11a8 admin admin] PATH_INFO: `/v3/projects/20bbd9ef0f234d3da52d2445122237b4/users/c4ed42b1681e4a5b8c1afe867701c77a/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:31.728994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bd22c-2e05-43d9-a898-1e5954fb11a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20bbd9ef0f234d3da52d2445122237b4, user_id=c4ed42b1681e4a5b8c1afe867701c77a, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:31.731834 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bd22c-2e05-43d9-a898-1e5954fb11a8 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:31.738075 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f064f7b5-d2f4-41ec-9211-e0475d516147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e56b89c86d344bc99d5860c8ac2e95ed'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:31.744931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 974/1947] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:31 2022] DELETE /identity/v3/users/e56b89c86d344bc99d5860c8ac2e95ed => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:31.748161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 974/1948] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:31 2022] PUT /identity/v3/projects/20bbd9ef0f234d3da52d2445122237b4/users/c4ed42b1681e4a5b8c1afe867701c77a/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:31.750366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a57f0b-bdef-45f6-a31a-e96881e5dbee None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:31.751034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a57f0b-bdef-45f6-a31a-e96881e5dbee None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:31.751483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a57f0b-bdef-45f6-a31a-e96881e5dbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:31.759389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1c021-874a-4803-894e-57c14b1078a1 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:31.759602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1c021-874a-4803-894e-57c14b1078a1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:31.759718 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1c021-874a-4803-894e-57c14b1078a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:31.771405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-28a57f0b-bdef-45f6-a31a-e96881e5dbee None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:31.772623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-28a57f0b-bdef-45f6-a31a-e96881e5dbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:31.797486 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-12e1c021-874a-4803-894e-57c14b1078a1 None None] MFA Rules not processed for user `c4ed42b1681e4a5b8c1afe867701c77a`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:31.799619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-12e1c021-874a-4803-894e-57c14b1078a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:31.845894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 975/1949] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:31.855765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a57f0b-bdef-45f6-a31a-e96881e5dbee None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:31.857120 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-28a57f0b-bdef-45f6-a31a-e96881e5dbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:31.884005 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 975/1950] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:31 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:31.896153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1c021-874a-4803-894e-57c14b1078a1 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:31.897526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-12e1c021-874a-4803-894e-57c14b1078a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:31.917524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a57f0b-bdef-45f6-a31a-e96881e5dbee None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:31.921524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd234ab-ba08-419e-9d32-f204ea7b4a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:31.922178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd234ab-ba08-419e-9d32-f204ea7b4a8d service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:31.924949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd234ab-ba08-419e-9d32-f204ea7b4a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:31.924949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd234ab-ba08-419e-9d32-f204ea7b4a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:31.933034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd234ab-ba08-419e-9d32-f204ea7b4a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:31.934477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd234ab-ba08-419e-9d32-f204ea7b4a8d service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:31.946050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 976/1951] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:31.959995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1c021-874a-4803-894e-57c14b1078a1 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:31.964132 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a31b63-7c15-466e-8d50-a711137325f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:31.966099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a31b63-7c15-466e-8d50-a711137325f3 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:31.966099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a31b63-7c15-466e-8d50-a711137325f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:31.966099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a31b63-7c15-466e-8d50-a711137325f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:31.976652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a31b63-7c15-466e-8d50-a711137325f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:31.978123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a31b63-7c15-466e-8d50-a711137325f3 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:31.988798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 976/1952] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:31 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:32.450844 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd234ab-ba08-419e-9d32-f204ea7b4a8d service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:32.452430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7bd234ab-ba08-419e-9d32-f204ea7b4a8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:32.465462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd234ab-ba08-419e-9d32-f204ea7b4a8d service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:32.469163 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380713e-2164-4732-8cf8-4de82d5dc9d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:32.469814 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380713e-2164-4732-8cf8-4de82d5dc9d2 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:32.470923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380713e-2164-4732-8cf8-4de82d5dc9d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:32.470923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380713e-2164-4732-8cf8-4de82d5dc9d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:32.472760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2380713e-2164-4732-8cf8-4de82d5dc9d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:32.518142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a31b63-7c15-466e-8d50-a711137325f3 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:32.519802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-15a31b63-7c15-466e-8d50-a711137325f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:32.534514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a31b63-7c15-466e-8d50-a711137325f3 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:32.541234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:32.542715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:32.543190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:32.543639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] PATH_INFO: `/v3/projects/1c0e6d5633a94767b98471703630972c` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:32.544919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380713e-2164-4732-8cf8-4de82d5dc9d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:32.546413 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380713e-2164-4732-8cf8-4de82d5dc9d2 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:32.546671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c0e6d5633a94767b98471703630972c)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:32.549466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:32.560114 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 977/1953] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:32 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:32.683817 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c0e6d5633a94767b98471703630972c', 'request_id': 'req-a9d585ad-bbac-4776-9b43-6339207fc208'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:32.794421 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c0e6d5633a94767b98471703630972c', 'request_id': 'req-a9d585ad-bbac-4776-9b43-6339207fc208'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:32.802545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c0e6d5633a94767b98471703630972c', 'request_id': 'req-a9d585ad-bbac-4776-9b43-6339207fc208'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:32.813618 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] The token cache is being invalidate because project 1c0e6d5633a94767b98471703630972c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:32.814205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:32.814672 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:32.816346 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 977/1954] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:32 2022] DELETE /identity/v3/projects/1c0e6d5633a94767b98471703630972c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:32.825511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380713e-2164-4732-8cf8-4de82d5dc9d2 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:32.839802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380713e-2164-4732-8cf8-4de82d5dc9d2 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:32.901458 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2f052-a364-4171-98a4-6047f28ac021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:32.902339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2f052-a364-4171-98a4-6047f28ac021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:32.902965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2f052-a364-4171-98a4-6047f28ac021 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:32.903242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2f052-a364-4171-98a4-6047f28ac021 admin admin] PATH_INFO: `/v3/users/723005fe72c64be3a96ee1c283364058` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:32.904438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2f052-a364-4171-98a4-6047f28ac021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=723005fe72c64be3a96ee1c283364058)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:32.906368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2f052-a364-4171-98a4-6047f28ac021 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:33.112252 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-76d2f052-a364-4171-98a4-6047f28ac021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '723005fe72c64be3a96ee1c283364058'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:33.121187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-76d2f052-a364-4171-98a4-6047f28ac021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '723005fe72c64be3a96ee1c283364058'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:33.124941 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-76d2f052-a364-4171-98a4-6047f28ac021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '723005fe72c64be3a96ee1c283364058'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:33.138469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-76d2f052-a364-4171-98a4-6047f28ac021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '723005fe72c64be3a96ee1c283364058'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:33.231597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:33.232495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-76d2f052-a364-4171-98a4-6047f28ac021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '723005fe72c64be3a96ee1c283364058'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:33.245481 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 978/1955] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:32 2022] DELETE /identity/v3/users/723005fe72c64be3a96ee1c283364058 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:33.546992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:34.432591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:34.434633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:34.452601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d585ad-bbac-4776-9b43-6339207fc208 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:34.572378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5f704-f223-4599-85d1-030c83edf915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:34.574061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5f704-f223-4599-85d1-030c83edf915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:34.574348 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5f704-f223-4599-85d1-030c83edf915 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:34.574626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5f704-f223-4599-85d1-030c83edf915 admin admin] PATH_INFO: `/v3/projects/874654f68c3944ce8ad75801c70ddcd8` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:34.575943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba5f704-f223-4599-85d1-030c83edf915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=874654f68c3944ce8ad75801c70ddcd8)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:34.578022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba5f704-f223-4599-85d1-030c83edf915 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:34.733332 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3ba5f704-f223-4599-85d1-030c83edf915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '874654f68c3944ce8ad75801c70ddcd8', 'request_id': 'req-3ba5f704-f223-4599-85d1-030c83edf915'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:34.886118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3ba5f704-f223-4599-85d1-030c83edf915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '874654f68c3944ce8ad75801c70ddcd8', 'request_id': 'req-3ba5f704-f223-4599-85d1-030c83edf915'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:34.891280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3ba5f704-f223-4599-85d1-030c83edf915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '874654f68c3944ce8ad75801c70ddcd8', 'request_id': 'req-3ba5f704-f223-4599-85d1-030c83edf915'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:34.904001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3ba5f704-f223-4599-85d1-030c83edf915 admin admin] The token cache is being invalidate because project 874654f68c3944ce8ad75801c70ddcd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:34.904584 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3ba5f704-f223-4599-85d1-030c83edf915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:34.905424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3ba5f704-f223-4599-85d1-030c83edf915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:34.908513 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 978/1956] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:34 2022] DELETE /identity/v3/projects/874654f68c3944ce8ad75801c70ddcd8 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:34.912649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527873b4-c74e-4f70-ae95-65a70f0a4755 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:34.913141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527873b4-c74e-4f70-ae95-65a70f0a4755 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:34.913141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527873b4-c74e-4f70-ae95-65a70f0a4755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:34.948474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-527873b4-c74e-4f70-ae95-65a70f0a4755 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:34.949754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-527873b4-c74e-4f70-ae95-65a70f0a4755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:35.030490 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 979/1957] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:35.038717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38a16f-e73a-4918-83b9-05cbed237c25 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:35.039069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38a16f-e73a-4918-83b9-05cbed237c25 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:35.039442 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38a16f-e73a-4918-83b9-05cbed237c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:35.059870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-0e38a16f-e73a-4918-83b9-05cbed237c25 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:35.061499 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0e38a16f-e73a-4918-83b9-05cbed237c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:35.070412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 979/1958] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:35.074691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527873b4-c74e-4f70-ae95-65a70f0a4755 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:35.111782 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527873b4-c74e-4f70-ae95-65a70f0a4755 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:35.120579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504bc31-9149-4d9e-97c2-52bcfdcc1e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:35.125700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504bc31-9149-4d9e-97c2-52bcfdcc1e21 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:35.125700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504bc31-9149-4d9e-97c2-52bcfdcc1e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:35.125700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504bc31-9149-4d9e-97c2-52bcfdcc1e21 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:35.125700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504bc31-9149-4d9e-97c2-52bcfdcc1e21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:35.125700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504bc31-9149-4d9e-97c2-52bcfdcc1e21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:35.125700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504bc31-9149-4d9e-97c2-52bcfdcc1e21 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:35.148381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38a16f-e73a-4918-83b9-05cbed237c25 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:35.156693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 980/1959] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:35.159670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38a16f-e73a-4918-83b9-05cbed237c25 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:35.164874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401dbce-11db-4a8f-b7da-23ab400a729b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:35.165565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401dbce-11db-4a8f-b7da-23ab400a729b admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:35.165849 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401dbce-11db-4a8f-b7da-23ab400a729b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:35.166124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401dbce-11db-4a8f-b7da-23ab400a729b admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:35.166852 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401dbce-11db-4a8f-b7da-23ab400a729b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:35.168864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401dbce-11db-4a8f-b7da-23ab400a729b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:35.182265 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:35.271244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 980/1960] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:35 2022] POST /identity/v3/projects => generated 388 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:35.272581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504bc31-9149-4d9e-97c2-52bcfdcc1e21 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:35.284137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504bc31-9149-4d9e-97c2-52bcfdcc1e21 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:35.351440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ac9cc-a174-4cb3-a995-277f4850a249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:35.352128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ac9cc-a174-4cb3-a995-277f4850a249 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:35.352653 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ac9cc-a174-4cb3-a995-277f4850a249 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:35.352846 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ac9cc-a174-4cb3-a995-277f4850a249 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:35.353917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ac9cc-a174-4cb3-a995-277f4850a249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:35.357098 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ac9cc-a174-4cb3-a995-277f4850a249 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:35.444484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401dbce-11db-4a8f-b7da-23ab400a729b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:35.446786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3401dbce-11db-4a8f-b7da-23ab400a729b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:35.462506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401dbce-11db-4a8f-b7da-23ab400a729b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:35.468070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21addb-e44f-485f-bfa1-01d735481d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:35.469522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21addb-e44f-485f-bfa1-01d735481d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:35.469907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21addb-e44f-485f-bfa1-01d735481d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:35.470669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21addb-e44f-485f-bfa1-01d735481d6f admin admin] PATH_INFO: `/v3/users/5a5dd73d80644aeca340746edae638eb` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:35.472144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21addb-e44f-485f-bfa1-01d735481d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a5dd73d80644aeca340746edae638eb)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:35.474282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21addb-e44f-485f-bfa1-01d735481d6f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:35.490225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 981/1961] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:35 2022] POST /identity/v3/users => generated 295 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:35.494914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ac9cc-a174-4cb3-a995-277f4850a249 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:35.513511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ac9cc-a174-4cb3-a995-277f4850a249 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:35.521169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff93590-1a76-4ec6-8b71-25e517938706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:35.521922 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff93590-1a76-4ec6-8b71-25e517938706 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:35.522267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff93590-1a76-4ec6-8b71-25e517938706 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:35.522579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff93590-1a76-4ec6-8b71-25e517938706 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:35.523350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff93590-1a76-4ec6-8b71-25e517938706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:35.525078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff93590-1a76-4ec6-8b71-25e517938706 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:35.557286 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 982/1962] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:35 2022] GET /identity/v3/roles => generated 1427 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:35.558407 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff93590-1a76-4ec6-8b71-25e517938706 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:35.569764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff93590-1a76-4ec6-8b71-25e517938706 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:35.575519 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c229e-dd15-474b-a98f-960852f4d352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:35.583192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c229e-dd15-474b-a98f-960852f4d352 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:35.583192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c229e-dd15-474b-a98f-960852f4d352 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:35.583192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c229e-dd15-474b-a98f-960852f4d352 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:35.583192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c229e-dd15-474b-a98f-960852f4d352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:35.583192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c229e-dd15-474b-a98f-960852f4d352 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:35.594522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 983/1963] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:35 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:35.596259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c229e-dd15-474b-a98f-960852f4d352 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:35.607646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c229e-dd15-474b-a98f-960852f4d352 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:35.612847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c810947-1805-47e0-9757-b8b847c922ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:35.613793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c810947-1805-47e0-9757-b8b847c922ad admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:35.614067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c810947-1805-47e0-9757-b8b847c922ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:35.614336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c810947-1805-47e0-9757-b8b847c922ad admin admin] PATH_INFO: `/v3/projects/233952adaddb4afd905d0a3f5be90b32/users/a6a66d2f52d44be392fc5c7ae320fecd/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:35.631155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c810947-1805-47e0-9757-b8b847c922ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=233952adaddb4afd905d0a3f5be90b32, user_id=a6a66d2f52d44be392fc5c7ae320fecd, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:35.633802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c810947-1805-47e0-9757-b8b847c922ad admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:35.709293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3c21addb-e44f-485f-bfa1-01d735481d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a5dd73d80644aeca340746edae638eb'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:35.740146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3c21addb-e44f-485f-bfa1-01d735481d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a5dd73d80644aeca340746edae638eb'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:35.740953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3c21addb-e44f-485f-bfa1-01d735481d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a5dd73d80644aeca340746edae638eb'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:35.760130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3c21addb-e44f-485f-bfa1-01d735481d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a5dd73d80644aeca340746edae638eb'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:35.771487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 984/1964] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:35 2022] PUT /identity/v3/projects/233952adaddb4afd905d0a3f5be90b32/users/a6a66d2f52d44be392fc5c7ae320fecd/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:35.783470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a86523-9c67-4851-9b10-dce0d4bfa002 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:35.783746 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a86523-9c67-4851-9b10-dce0d4bfa002 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:35.784021 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a86523-9c67-4851-9b10-dce0d4bfa002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:35.792709 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:35.830990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-38a86523-9c67-4851-9b10-dce0d4bfa002 None None] MFA Rules not processed for user `a6a66d2f52d44be392fc5c7ae320fecd`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:35.832423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-38a86523-9c67-4851-9b10-dce0d4bfa002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:35.852218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-3c21addb-e44f-485f-bfa1-01d735481d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a5dd73d80644aeca340746edae638eb'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:35.859036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 981/1965] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:35 2022] DELETE /identity/v3/users/5a5dd73d80644aeca340746edae638eb => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:35.870048 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba71a-aef5-4ed0-ad7c-9fc5793d7d6c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:35.870513 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba71a-aef5-4ed0-ad7c-9fc5793d7d6c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:35.870941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba71a-aef5-4ed0-ad7c-9fc5793d7d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:35.910787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-11fba71a-aef5-4ed0-ad7c-9fc5793d7d6c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:35.912489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-11fba71a-aef5-4ed0-ad7c-9fc5793d7d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:35.926852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 985/1966] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:35.930652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a86523-9c67-4851-9b10-dce0d4bfa002 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:35.932230 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-38a86523-9c67-4851-9b10-dce0d4bfa002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:36.016515 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 982/1967] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:36.025298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fba71a-aef5-4ed0-ad7c-9fc5793d7d6c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:36.027406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-11fba71a-aef5-4ed0-ad7c-9fc5793d7d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:36.070854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a86523-9c67-4851-9b10-dce0d4bfa002 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:36.078392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353df983-1b0f-4358-9a1e-b57ab3c818f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:36.080536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353df983-1b0f-4358-9a1e-b57ab3c818f5 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:36.080840 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353df983-1b0f-4358-9a1e-b57ab3c818f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:36.081122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353df983-1b0f-4358-9a1e-b57ab3c818f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:36.083635 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353df983-1b0f-4358-9a1e-b57ab3c818f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:36.085931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353df983-1b0f-4358-9a1e-b57ab3c818f5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:36.102035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fba71a-aef5-4ed0-ad7c-9fc5793d7d6c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:36.106257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7e909-b386-4ac2-b948-18010b64eed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:36.107034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7e909-b386-4ac2-b948-18010b64eed0 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:36.107390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7e909-b386-4ac2-b948-18010b64eed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:36.107736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7e909-b386-4ac2-b948-18010b64eed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:36.121398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7e909-b386-4ac2-b948-18010b64eed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:36.123446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7e909-b386-4ac2-b948-18010b64eed0 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:36.134856 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 983/1968] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:36.199287 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 986/1969] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:35 2022] POST /identity/v3/projects => generated 386 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:36.202506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7e909-b386-4ac2-b948-18010b64eed0 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:36.218651 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7e909-b386-4ac2-b948-18010b64eed0 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:36.300943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fadaba-9dd7-49b0-b0b4-95f94eeb9c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:36.301619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fadaba-9dd7-49b0-b0b4-95f94eeb9c0f admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:36.301900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fadaba-9dd7-49b0-b0b4-95f94eeb9c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:36.302170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fadaba-9dd7-49b0-b0b4-95f94eeb9c0f admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:36.303590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fadaba-9dd7-49b0-b0b4-95f94eeb9c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:36.305014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fadaba-9dd7-49b0-b0b4-95f94eeb9c0f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:36.424425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 984/1970] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:36 2022] POST /identity/v3/users => generated 300 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:36.428398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353df983-1b0f-4358-9a1e-b57ab3c818f5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:36.439761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353df983-1b0f-4358-9a1e-b57ab3c818f5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:36.445776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1543c-b713-41ba-b89f-d79f1e535320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:36.446464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1543c-b713-41ba-b89f-d79f1e535320 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:36.446744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1543c-b713-41ba-b89f-d79f1e535320 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:36.447048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1543c-b713-41ba-b89f-d79f1e535320 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:36.447808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1543c-b713-41ba-b89f-d79f1e535320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:36.449572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1543c-b713-41ba-b89f-d79f1e535320 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:36.469836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fadaba-9dd7-49b0-b0b4-95f94eeb9c0f admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:36.471954 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 987/1971] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:36 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:36.478507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fadaba-9dd7-49b0-b0b4-95f94eeb9c0f admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:36.483343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552ad3e-79f1-4a1d-9275-a7e86922b8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:36.484245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552ad3e-79f1-4a1d-9275-a7e86922b8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:36.484703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552ad3e-79f1-4a1d-9275-a7e86922b8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:36.484984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552ad3e-79f1-4a1d-9275-a7e86922b8a2 admin admin] PATH_INFO: `/v3/projects/9bbcd98ba2c24e6aaee680d1e49b9847/users/4269ee9d4c9547b7b7f15f854b9d0094/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:36.502515 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552ad3e-79f1-4a1d-9275-a7e86922b8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bbcd98ba2c24e6aaee680d1e49b9847, user_id=4269ee9d4c9547b7b7f15f854b9d0094, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:36.505404 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552ad3e-79f1-4a1d-9275-a7e86922b8a2 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:36.649375 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 985/1972] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:36 2022] PUT /identity/v3/projects/9bbcd98ba2c24e6aaee680d1e49b9847/users/4269ee9d4c9547b7b7f15f854b9d0094/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:36.653778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1543c-b713-41ba-b89f-d79f1e535320 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:36.667382 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1543c-b713-41ba-b89f-d79f1e535320 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:36.744496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff044f5-9f1b-4d90-a904-9a754c909526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:36.745208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff044f5-9f1b-4d90-a904-9a754c909526 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:36.745583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff044f5-9f1b-4d90-a904-9a754c909526 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:36.745872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff044f5-9f1b-4d90-a904-9a754c909526 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:36.746866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff044f5-9f1b-4d90-a904-9a754c909526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:36.748473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff044f5-9f1b-4d90-a904-9a754c909526 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:36.759069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 988/1973] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:36 2022] GET /identity/v3/roles => generated 1427 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:36.763377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552ad3e-79f1-4a1d-9275-a7e86922b8a2 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:36.768564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff044f5-9f1b-4d90-a904-9a754c909526 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:36.768853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-2552ad3e-79f1-4a1d-9275-a7e86922b8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:36.777022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff044f5-9f1b-4d90-a904-9a754c909526 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:36.782126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e8724-bdb3-484a-a2f8-898395719507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:36.783671 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e8724-bdb3-484a-a2f8-898395719507 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:36.783956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e8724-bdb3-484a-a2f8-898395719507 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:36.784211 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e8724-bdb3-484a-a2f8-898395719507 admin admin] PATH_INFO: `/v3/projects/9bbcd98ba2c24e6aaee680d1e49b9847/users/4269ee9d4c9547b7b7f15f854b9d0094/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:36.786243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e8724-bdb3-484a-a2f8-898395719507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bbcd98ba2c24e6aaee680d1e49b9847, user_id=4269ee9d4c9547b7b7f15f854b9d0094, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:36.787892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552ad3e-79f1-4a1d-9275-a7e86922b8a2 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:36.789332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e8724-bdb3-484a-a2f8-898395719507 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:36.794446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:36.795354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:36.795907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:36.796185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] PATH_INFO: `/v3/projects/1482d4c5ea0941b795e12d7879e256f0` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:36.797575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1482d4c5ea0941b795e12d7879e256f0)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:36.799759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:36.803082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 989/1974] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:36 2022] PUT /identity/v3/projects/9bbcd98ba2c24e6aaee680d1e49b9847/users/4269ee9d4c9547b7b7f15f854b9d0094/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:36.813426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a499e47-7b99-4ff4-b4f6-f71eba3c679e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:36.813743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a499e47-7b99-4ff4-b4f6-f71eba3c679e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:36.814475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a499e47-7b99-4ff4-b4f6-f71eba3c679e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:36.846119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9a499e47-7b99-4ff4-b4f6-f71eba3c679e None None] MFA Rules not processed for user `4269ee9d4c9547b7b7f15f854b9d0094`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:36.847349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9a499e47-7b99-4ff4-b4f6-f71eba3c679e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:36.916827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 990/1975] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:36.927739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a499e47-7b99-4ff4-b4f6-f71eba3c679e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:36.929150 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9a499e47-7b99-4ff4-b4f6-f71eba3c679e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:36.941943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1482d4c5ea0941b795e12d7879e256f0', 'request_id': 'req-582eea11-d0fb-471b-907a-cfc40a6c685d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:36.989213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a499e47-7b99-4ff4-b4f6-f71eba3c679e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:37.020145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1482d4c5ea0941b795e12d7879e256f0', 'request_id': 'req-582eea11-d0fb-471b-907a-cfc40a6c685d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:37.028963 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1482d4c5ea0941b795e12d7879e256f0', 'request_id': 'req-582eea11-d0fb-471b-907a-cfc40a6c685d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:37.041562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] The token cache is being invalidate because project 1482d4c5ea0941b795e12d7879e256f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:37.042212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:37.043124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:37.047913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71311321-827d-4deb-850b-ce926efa3259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:37.048213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 986/1976] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:36 2022] DELETE /identity/v3/projects/1482d4c5ea0941b795e12d7879e256f0 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:37.048325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71311321-827d-4deb-850b-ce926efa3259 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:37.048325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71311321-827d-4deb-850b-ce926efa3259 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:37.048325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71311321-827d-4deb-850b-ce926efa3259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:37.049352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:37.050301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-71311321-827d-4deb-850b-ce926efa3259 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:37.050732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:37.121406 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71311321-827d-4deb-850b-ce926efa3259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:37.122973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71311321-827d-4deb-850b-ce926efa3259 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:37.133947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582eea11-d0fb-471b-907a-cfc40a6c685d admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:37.134328 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 991/1977] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:37.138548 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae5c37-898f-499d-baa9-df1d9c79f06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:37.139471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae5c37-898f-499d-baa9-df1d9c79f06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:37.139745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae5c37-898f-499d-baa9-df1d9c79f06d admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:37.140049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae5c37-898f-499d-baa9-df1d9c79f06d admin admin] PATH_INFO: `/v3/users/45eb46fb5c314bf389ba5e7e40385452` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:37.141197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae5c37-898f-499d-baa9-df1d9c79f06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=45eb46fb5c314bf389ba5e7e40385452)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:37.143415 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae5c37-898f-499d-baa9-df1d9c79f06d admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:37.378265 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-c3ae5c37-898f-499d-baa9-df1d9c79f06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45eb46fb5c314bf389ba5e7e40385452'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:37.385773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-c3ae5c37-898f-499d-baa9-df1d9c79f06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45eb46fb5c314bf389ba5e7e40385452'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:37.386491 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-c3ae5c37-898f-499d-baa9-df1d9c79f06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45eb46fb5c314bf389ba5e7e40385452'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:37.402310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-c3ae5c37-898f-499d-baa9-df1d9c79f06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45eb46fb5c314bf389ba5e7e40385452'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:37.543659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-c3ae5c37-898f-499d-baa9-df1d9c79f06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45eb46fb5c314bf389ba5e7e40385452'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:37.551124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 987/1978] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:37 2022] DELETE /identity/v3/users/45eb46fb5c314bf389ba5e7e40385452 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:38.298067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71311321-827d-4deb-850b-ce926efa3259 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:38.307586 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71311321-827d-4deb-850b-ce926efa3259 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:38.471451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59cf970-b9f4-4570-b10f-9d7187c8e302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:38.471451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cf970-b9f4-4570-b10f-9d7187c8e302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:38.471451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cf970-b9f4-4570-b10f-9d7187c8e302 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:38.471451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cf970-b9f4-4570-b10f-9d7187c8e302 admin admin] PATH_INFO: `/v3/projects/43205e3676e64b0eb4dd4d0eb382e5e7` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:38.472308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59cf970-b9f4-4570-b10f-9d7187c8e302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43205e3676e64b0eb4dd4d0eb382e5e7)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:38.472308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59cf970-b9f4-4570-b10f-9d7187c8e302 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:38.774508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e59cf970-b9f4-4570-b10f-9d7187c8e302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43205e3676e64b0eb4dd4d0eb382e5e7', 'request_id': 'req-e59cf970-b9f4-4570-b10f-9d7187c8e302'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:38.889168 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e59cf970-b9f4-4570-b10f-9d7187c8e302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43205e3676e64b0eb4dd4d0eb382e5e7', 'request_id': 'req-e59cf970-b9f4-4570-b10f-9d7187c8e302'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:38.896406 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e59cf970-b9f4-4570-b10f-9d7187c8e302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43205e3676e64b0eb4dd4d0eb382e5e7', 'request_id': 'req-e59cf970-b9f4-4570-b10f-9d7187c8e302'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:38.907028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e59cf970-b9f4-4570-b10f-9d7187c8e302 admin admin] The token cache is being invalidate because project 43205e3676e64b0eb4dd4d0eb382e5e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:38.907319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e59cf970-b9f4-4570-b10f-9d7187c8e302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:38.907952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e59cf970-b9f4-4570-b10f-9d7187c8e302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:38.910208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 992/1979] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:38 2022] DELETE /identity/v3/projects/43205e3676e64b0eb4dd4d0eb382e5e7 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:38.916295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dce98-e3d0-4634-a157-f23902826d24 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:38.916832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dce98-e3d0-4634-a157-f23902826d24 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:38.916933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dce98-e3d0-4634-a157-f23902826d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:38.935263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-024dce98-e3d0-4634-a157-f23902826d24 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:38.936625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-024dce98-e3d0-4634-a157-f23902826d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:38.997792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 988/1980] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:39.005950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21eba5-764e-4aa2-a5fc-0df972e2fc40 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:39.006323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21eba5-764e-4aa2-a5fc-0df972e2fc40 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:39.006676 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21eba5-764e-4aa2-a5fc-0df972e2fc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:39.027561 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9e21eba5-764e-4aa2-a5fc-0df972e2fc40 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:39.029070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9e21eba5-764e-4aa2-a5fc-0df972e2fc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:39.032780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dce98-e3d0-4634-a157-f23902826d24 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:39.035238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-024dce98-e3d0-4634-a157-f23902826d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:39.039554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 993/1981] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:39.046219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21eba5-764e-4aa2-a5fc-0df972e2fc40 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:39.050603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dce98-e3d0-4634-a157-f23902826d24 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:39.055444 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21eba5-764e-4aa2-a5fc-0df972e2fc40 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:39.055831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:39.056607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:39.056889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:39.057266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] PATH_INFO: `/v3/users/a6a66d2f52d44be392fc5c7ae320fecd` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:39.058481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6a66d2f52d44be392fc5c7ae320fecd)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:39.060530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:39.060921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16c855-4eea-4a88-a53d-07a679ed7577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:39.061622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16c855-4eea-4a88-a53d-07a679ed7577 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:39.061926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16c855-4eea-4a88-a53d-07a679ed7577 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:39.062205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16c855-4eea-4a88-a53d-07a679ed7577 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:39.062999 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16c855-4eea-4a88-a53d-07a679ed7577 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:39.063323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16c855-4eea-4a88-a53d-07a679ed7577 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:39.067042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16c855-4eea-4a88-a53d-07a679ed7577 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:39.081683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 994/1982] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:00:39.086349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16c855-4eea-4a88-a53d-07a679ed7577 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:39.097265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16c855-4eea-4a88-a53d-07a679ed7577 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:39.105729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df235ed0-9492-40bf-b089-882cd58f4629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:39.110763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:39.111470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df235ed0-9492-40bf-b089-882cd58f4629 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:39.111470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df235ed0-9492-40bf-b089-882cd58f4629 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:39.111470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df235ed0-9492-40bf-b089-882cd58f4629 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:39.111470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df235ed0-9492-40bf-b089-882cd58f4629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:39.111470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df235ed0-9492-40bf-b089-882cd58f4629 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:39.211116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 995/1983] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:39 2022] POST /identity/v3/projects => generated 368 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:39.215291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df235ed0-9492-40bf-b089-882cd58f4629 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:39.279995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df235ed0-9492-40bf-b089-882cd58f4629 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:39.286018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b916482-9e4c-4aa1-96b7-5b05478d8e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:39.286736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b916482-9e4c-4aa1-96b7-5b05478d8e16 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:39.287064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b916482-9e4c-4aa1-96b7-5b05478d8e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:39.287178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6a66d2f52d44be392fc5c7ae320fecd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:39.287375 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b916482-9e4c-4aa1-96b7-5b05478d8e16 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:39.288078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b916482-9e4c-4aa1-96b7-5b05478d8e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:39.290032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b916482-9e4c-4aa1-96b7-5b05478d8e16 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:39.294284 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6a66d2f52d44be392fc5c7ae320fecd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:39.294975 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6a66d2f52d44be392fc5c7ae320fecd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:39.308036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6a66d2f52d44be392fc5c7ae320fecd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:39.400770 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6a66d2f52d44be392fc5c7ae320fecd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:39.407333 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 996/1984] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:39 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:39.411399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 989/1985] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:39 2022] DELETE /identity/v3/users/a6a66d2f52d44be392fc5c7ae320fecd => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:39.412303 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:39.412866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15dcdfa-5691-4707-9ce5-1cfc2316aa45 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:39.413162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15dcdfa-5691-4707-9ce5-1cfc2316aa45 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:39.413443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15dcdfa-5691-4707-9ce5-1cfc2316aa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:39.413749 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:39.433165 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e15dcdfa-5691-4707-9ce5-1cfc2316aa45 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:39.434396 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e15dcdfa-5691-4707-9ce5-1cfc2316aa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:39.490482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b6ccf-c5fd-4bab-aa41-16efd1f8fe1b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:39.495309 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c88ec5-0791-43df-a176-75f89678b33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:39.495986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c88ec5-0791-43df-a176-75f89678b33c admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:39.496268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c88ec5-0791-43df-a176-75f89678b33c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:39.496563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c88ec5-0791-43df-a176-75f89678b33c admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:39.497967 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c88ec5-0791-43df-a176-75f89678b33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:39.498141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:39.499596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c88ec5-0791-43df-a176-75f89678b33c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:39.500917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 997/1986] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:39.510125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 990/1987] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:39 2022] GET /identity/v3/roles => generated 1427 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:39.514369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15dcdfa-5691-4707-9ce5-1cfc2316aa45 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:39.514529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c88ec5-0791-43df-a176-75f89678b33c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:39.516205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-18c88ec5-0791-43df-a176-75f89678b33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:39.525291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15dcdfa-5691-4707-9ce5-1cfc2316aa45 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:39.531868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88cbb0-37ae-4093-a8bb-8e85693b0fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:39.532544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88cbb0-37ae-4093-a8bb-8e85693b0fdd admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:39.532825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88cbb0-37ae-4093-a8bb-8e85693b0fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:39.533143 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88cbb0-37ae-4093-a8bb-8e85693b0fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:39.534410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88cbb0-37ae-4093-a8bb-8e85693b0fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:39.536065 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88cbb0-37ae-4093-a8bb-8e85693b0fdd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:39.546274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 998/1988] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:39 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:39.550968 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88cbb0-37ae-4093-a8bb-8e85693b0fdd admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:39.560449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88cbb0-37ae-4093-a8bb-8e85693b0fdd admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:39.566079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7574118-2161-41a1-b95e-59fb7c1e7a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:39.567013 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7574118-2161-41a1-b95e-59fb7c1e7a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:39.567293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7574118-2161-41a1-b95e-59fb7c1e7a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:39.567567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7574118-2161-41a1-b95e-59fb7c1e7a21 admin admin] PATH_INFO: `/v3/projects/f81fb10777f541178958502badde88bc/users/f232a5374b71455b8efb4f5e3bf593d7/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:39.576819 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c88ec5-0791-43df-a176-75f89678b33c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:39.581130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403625b-9e68-40c4-a5a7-5832a82785e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:39.583825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403625b-9e68-40c4-a5a7-5832a82785e0 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:39.583825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403625b-9e68-40c4-a5a7-5832a82785e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:39.583825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403625b-9e68-40c4-a5a7-5832a82785e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:39.585599 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7574118-2161-41a1-b95e-59fb7c1e7a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81fb10777f541178958502badde88bc, user_id=f232a5374b71455b8efb4f5e3bf593d7, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:39.588608 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7574118-2161-41a1-b95e-59fb7c1e7a21 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:39.592433 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403625b-9e68-40c4-a5a7-5832a82785e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:39.593884 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403625b-9e68-40c4-a5a7-5832a82785e0 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:39.606482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 991/1989] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:39.747308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 999/1990] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:39 2022] PUT /identity/v3/projects/f81fb10777f541178958502badde88bc/users/f232a5374b71455b8efb4f5e3bf593d7/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:39.763841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b01bd-8273-42af-9458-043b7287794b None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:39.764140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b01bd-8273-42af-9458-043b7287794b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:39.764425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b01bd-8273-42af-9458-043b7287794b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:39.794647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c24b01bd-8273-42af-9458-043b7287794b None None] MFA Rules not processed for user `f232a5374b71455b8efb4f5e3bf593d7`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:39.795909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c24b01bd-8273-42af-9458-043b7287794b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:39.896646 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 992/1991] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:39.901147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7574118-2161-41a1-b95e-59fb7c1e7a21 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:39.974851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7574118-2161-41a1-b95e-59fb7c1e7a21 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:39.981133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9e05f-b83d-4923-b357-e945b918fc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:39.981821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e05f-b83d-4923-b357-e945b918fc6d admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:39.982109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e05f-b83d-4923-b357-e945b918fc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:39.982386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e05f-b83d-4923-b357-e945b918fc6d admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:39.983196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9e05f-b83d-4923-b357-e945b918fc6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:39.985132 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9e05f-b83d-4923-b357-e945b918fc6d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:40.045296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b01bd-8273-42af-9458-043b7287794b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:40.047179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c24b01bd-8273-42af-9458-043b7287794b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:40.070007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b01bd-8273-42af-9458-043b7287794b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:40.074993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:40.076002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:40.076606 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:40.077153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] PATH_INFO: `/v3/projects/233952adaddb4afd905d0a3f5be90b32` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:40.078162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1000/1992] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:39 2022] POST /identity/v3/projects => generated 370 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:40.078278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=233952adaddb4afd905d0a3f5be90b32)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:40.079091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9e05f-b83d-4923-b357-e945b918fc6d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:40.080958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:40.142389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9e05f-b83d-4923-b357-e945b918fc6d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:40.147946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba242b61-6f29-4713-ba6b-9e5bd14526a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:40.149185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba242b61-6f29-4713-ba6b-9e5bd14526a5 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:40.149185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba242b61-6f29-4713-ba6b-9e5bd14526a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:40.149185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba242b61-6f29-4713-ba6b-9e5bd14526a5 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:40.149917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba242b61-6f29-4713-ba6b-9e5bd14526a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:40.151848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba242b61-6f29-4713-ba6b-9e5bd14526a5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:40.219489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233952adaddb4afd905d0a3f5be90b32', 'request_id': 'req-eae4c42b-984e-4074-b827-9e0ace4f49c1'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:40.249595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1001/1993] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:40 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:40.254636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba242b61-6f29-4713-ba6b-9e5bd14526a5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:40.266646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba242b61-6f29-4713-ba6b-9e5bd14526a5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:40.306279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233952adaddb4afd905d0a3f5be90b32', 'request_id': 'req-eae4c42b-984e-4074-b827-9e0ace4f49c1'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:40.313607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233952adaddb4afd905d0a3f5be90b32', 'request_id': 'req-eae4c42b-984e-4074-b827-9e0ace4f49c1'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:40.325442 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] The token cache is being invalidate because project 233952adaddb4afd905d0a3f5be90b32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:40.325691 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:40.326577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:40.327855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99611168-4185-4226-8c09-0d007730487d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:40.328541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99611168-4185-4226-8c09-0d007730487d admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:40.328943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99611168-4185-4226-8c09-0d007730487d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:40.329226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99611168-4185-4226-8c09-0d007730487d admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:40.329956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99611168-4185-4226-8c09-0d007730487d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:40.331630 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 993/1994] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:40 2022] DELETE /identity/v3/projects/233952adaddb4afd905d0a3f5be90b32 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:40.332430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:40.333170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99611168-4185-4226-8c09-0d007730487d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:40.333859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:40.343963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1002/1995] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:40 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:40.346929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4c42b-984e-4074-b827-9e0ace4f49c1 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:40.348555 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99611168-4185-4226-8c09-0d007730487d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:40.350325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-99611168-4185-4226-8c09-0d007730487d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:40.351826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:40.352655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:40.352939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:40.353213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] PATH_INFO: `/v3/users/4269ee9d4c9547b7b7f15f854b9d0094` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:40.354936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4269ee9d4c9547b7b7f15f854b9d0094)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:40.356987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:40.365359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99611168-4185-4226-8c09-0d007730487d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:40.373017 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68448471-c619-4cb6-ab9d-fb96c1e4e19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:40.373908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68448471-c619-4cb6-ab9d-fb96c1e4e19a admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:40.374192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68448471-c619-4cb6-ab9d-fb96c1e4e19a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:40.374470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68448471-c619-4cb6-ab9d-fb96c1e4e19a admin admin] PATH_INFO: `/v3/projects/cb70021a2fd94717840fa3fd00a7a7d0/users/a826f366ec57437eb50bd628d09801a6/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:40.393228 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68448471-c619-4cb6-ab9d-fb96c1e4e19a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb70021a2fd94717840fa3fd00a7a7d0, user_id=a826f366ec57437eb50bd628d09801a6, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:40.395992 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68448471-c619-4cb6-ab9d-fb96c1e4e19a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:40.539571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1003/1996] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:40 2022] PUT /identity/v3/projects/cb70021a2fd94717840fa3fd00a7a7d0/users/a826f366ec57437eb50bd628d09801a6/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:40.543369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68448471-c619-4cb6-ab9d-fb96c1e4e19a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:40.609404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68448471-c619-4cb6-ab9d-fb96c1e4e19a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:40.614980 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792f2d1-9b75-491c-8de1-d5882c6ad693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:40.615666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792f2d1-9b75-491c-8de1-d5882c6ad693 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:40.615949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792f2d1-9b75-491c-8de1-d5882c6ad693 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:40.616231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792f2d1-9b75-491c-8de1-d5882c6ad693 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:40.616949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792f2d1-9b75-491c-8de1-d5882c6ad693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:40.619073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792f2d1-9b75-491c-8de1-d5882c6ad693 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:40.621500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4269ee9d4c9547b7b7f15f854b9d0094'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:40.629570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4269ee9d4c9547b7b7f15f854b9d0094'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:40.633263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1004/1997] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:40 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:40.633424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4269ee9d4c9547b7b7f15f854b9d0094'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:40.634532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792f2d1-9b75-491c-8de1-d5882c6ad693 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:40.645164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792f2d1-9b75-491c-8de1-d5882c6ad693 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:40.645306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4269ee9d4c9547b7b7f15f854b9d0094'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:40.649850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca597298-f993-4eae-a56e-ee72cf161b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:40.650754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca597298-f993-4eae-a56e-ee72cf161b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:40.651076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca597298-f993-4eae-a56e-ee72cf161b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:40.651351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca597298-f993-4eae-a56e-ee72cf161b87 admin admin] PATH_INFO: `/v3/projects/cb70021a2fd94717840fa3fd00a7a7d0/users/a826f366ec57437eb50bd628d09801a6/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:40.653767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca597298-f993-4eae-a56e-ee72cf161b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb70021a2fd94717840fa3fd00a7a7d0, user_id=a826f366ec57437eb50bd628d09801a6, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:40.656304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca597298-f993-4eae-a56e-ee72cf161b87 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:40.670764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1005/1998] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:40 2022] PUT /identity/v3/projects/cb70021a2fd94717840fa3fd00a7a7d0/users/a826f366ec57437eb50bd628d09801a6/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:40.682349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b24e7-eee7-4116-8c96-f5f48fa4bfa8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:40.682697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b24e7-eee7-4116-8c96-f5f48fa4bfa8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:40.683028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b24e7-eee7-4116-8c96-f5f48fa4bfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:40.712115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4269ee9d4c9547b7b7f15f854b9d0094'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:40.712416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-129b24e7-eee7-4116-8c96-f5f48fa4bfa8 None None] MFA Rules not processed for user `a826f366ec57437eb50bd628d09801a6`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:40.713662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-129b24e7-eee7-4116-8c96-f5f48fa4bfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:40.719633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 994/1999] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:40 2022] DELETE /identity/v3/users/4269ee9d4c9547b7b7f15f854b9d0094 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:40.786542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1006/2000] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:40.797778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:40.800169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:40.858084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76bcf7-f68c-464d-99b9-f07394ed9b58 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:40.861799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b2fe7-f7c0-4446-ae60-b3b6d5fdfb8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:40.862455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b2fe7-f7c0-4446-ae60-b3b6d5fdfb8c service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:40.862737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b2fe7-f7c0-4446-ae60-b3b6d5fdfb8c service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:40.863078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b2fe7-f7c0-4446-ae60-b3b6d5fdfb8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:40.938609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b2fe7-f7c0-4446-ae60-b3b6d5fdfb8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:40.940697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b2fe7-f7c0-4446-ae60-b3b6d5fdfb8c service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:40.959845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 995/2001] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:41.077793 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b24e7-eee7-4116-8c96-f5f48fa4bfa8 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:41.147560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b24e7-eee7-4116-8c96-f5f48fa4bfa8 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:41.154161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b2fe7-f7c0-4446-ae60-b3b6d5fdfb8c service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:41.168177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937c943-6003-4968-9fbb-69ebac8d50cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:41.168740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c56b2fe7-f7c0-4446-ae60-b3b6d5fdfb8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:41.168829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937c943-6003-4968-9fbb-69ebac8d50cd service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:41.168829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937c943-6003-4968-9fbb-69ebac8d50cd service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:41.168829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937c943-6003-4968-9fbb-69ebac8d50cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:41.168829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4937c943-6003-4968-9fbb-69ebac8d50cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:41.240041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937c943-6003-4968-9fbb-69ebac8d50cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:41.240686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b2fe7-f7c0-4446-ae60-b3b6d5fdfb8c service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:41.241774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937c943-6003-4968-9fbb-69ebac8d50cd service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:41.245343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:41.246231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:41.246514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:41.246789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f admin admin] PATH_INFO: `/v3/projects/9bbcd98ba2c24e6aaee680d1e49b9847` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:41.248240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bbcd98ba2c24e6aaee680d1e49b9847)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:41.250390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:41.253496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1007/2002] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:41.306719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:41.450714 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bbcd98ba2c24e6aaee680d1e49b9847', 'request_id': 'req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:41.618282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bbcd98ba2c24e6aaee680d1e49b9847', 'request_id': 'req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:41.626074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bbcd98ba2c24e6aaee680d1e49b9847', 'request_id': 'req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:41.637177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f admin admin] The token cache is being invalidate because project 9bbcd98ba2c24e6aaee680d1e49b9847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:41.637483 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:41.638126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5d23bf63-2f3a-40e4-b623-78ecb5b7119f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:41.639714 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 996/2003] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:41 2022] DELETE /identity/v3/projects/9bbcd98ba2c24e6aaee680d1e49b9847 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:41.644831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8175e1d-0349-4a56-bdf0-3a0738258ac9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:41.645356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8175e1d-0349-4a56-bdf0-3a0738258ac9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:41.645665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8175e1d-0349-4a56-bdf0-3a0738258ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:41.666229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c8175e1d-0349-4a56-bdf0-3a0738258ac9 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:41.667835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c8175e1d-0349-4a56-bdf0-3a0738258ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:41.741851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1008/2004] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:41.749837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ce6f2-450d-44bf-a967-3e697f48005e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:41.750781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ce6f2-450d-44bf-a967-3e697f48005e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:41.751355 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ce6f2-450d-44bf-a967-3e697f48005e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:41.783101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b37ce6f2-450d-44bf-a967-3e697f48005e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:41.783101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b37ce6f2-450d-44bf-a967-3e697f48005e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:41.790352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 997/2005] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:41.795498 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8175e1d-0349-4a56-bdf0-3a0738258ac9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:41.807091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8175e1d-0349-4a56-bdf0-3a0738258ac9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:41.813759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163eb0b-f1ef-4482-9a92-f68c843f8546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:41.814449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163eb0b-f1ef-4482-9a92-f68c843f8546 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:41.815360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163eb0b-f1ef-4482-9a92-f68c843f8546 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:41.816114 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163eb0b-f1ef-4482-9a92-f68c843f8546 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:41.816564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163eb0b-f1ef-4482-9a92-f68c843f8546 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:41.816878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163eb0b-f1ef-4482-9a92-f68c843f8546 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:41.818506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163eb0b-f1ef-4482-9a92-f68c843f8546 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:41.831002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1009/2006] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:41.835659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ce6f2-450d-44bf-a967-3e697f48005e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:41.845390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ce6f2-450d-44bf-a967-3e697f48005e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:41.851216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5194f-734a-4c9a-8cd3-d17badc76edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:41.851898 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5194f-734a-4c9a-8cd3-d17badc76edc admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:41.852305 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5194f-734a-4c9a-8cd3-d17badc76edc admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:41.852575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5194f-734a-4c9a-8cd3-d17badc76edc admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:41.853378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5194f-734a-4c9a-8cd3-d17badc76edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:41.855523 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5194f-734a-4c9a-8cd3-d17badc76edc admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:42.003046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 998/2007] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:41 2022] POST /identity/v3/projects => generated 384 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:42.008054 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163eb0b-f1ef-4482-9a92-f68c843f8546 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:42.025686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163eb0b-f1ef-4482-9a92-f68c843f8546 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:42.028944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:42.116275 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e82b5-22d4-4697-af34-b11ea49c35f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:42.116996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e82b5-22d4-4697-af34-b11ea49c35f9 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:42.117277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e82b5-22d4-4697-af34-b11ea49c35f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:42.117545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e82b5-22d4-4697-af34-b11ea49c35f9 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:42.118250 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e82b5-22d4-4697-af34-b11ea49c35f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:42.120204 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e82b5-22d4-4697-af34-b11ea49c35f9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:42.204115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1010/2008] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:42 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:42.205874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5194f-734a-4c9a-8cd3-d17badc76edc admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:42.218685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5194f-734a-4c9a-8cd3-d17badc76edc admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:42.225783 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10ba81-6ae9-4a28-8f7e-786e567231f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:42.226511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10ba81-6ae9-4a28-8f7e-786e567231f5 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:42.226852 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10ba81-6ae9-4a28-8f7e-786e567231f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:42.227151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10ba81-6ae9-4a28-8f7e-786e567231f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:42.227914 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10ba81-6ae9-4a28-8f7e-786e567231f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:42.229627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10ba81-6ae9-4a28-8f7e-786e567231f5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:42.243002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 999/2009] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:42 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:42.251205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e82b5-22d4-4697-af34-b11ea49c35f9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:42.265134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e82b5-22d4-4697-af34-b11ea49c35f9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:42.273337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f49e8e-2f6f-4991-97f8-3304020c88e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:42.274379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f49e8e-2f6f-4991-97f8-3304020c88e7 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:42.274948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f49e8e-2f6f-4991-97f8-3304020c88e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:42.275387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f49e8e-2f6f-4991-97f8-3304020c88e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:42.280355 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f49e8e-2f6f-4991-97f8-3304020c88e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:42.280355 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f49e8e-2f6f-4991-97f8-3304020c88e7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:42.293216 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1011/2010] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:42 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:42.297336 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10ba81-6ae9-4a28-8f7e-786e567231f5 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:42.308370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10ba81-6ae9-4a28-8f7e-786e567231f5 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:42.312941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bc611-fc5e-4806-8fac-bdaf995eb036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:42.313833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bc611-fc5e-4806-8fac-bdaf995eb036 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:42.314112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bc611-fc5e-4806-8fac-bdaf995eb036 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:42.314395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bc611-fc5e-4806-8fac-bdaf995eb036 admin admin] PATH_INFO: `/v3/projects/4dbd55faf7a545119decc8f932782ae3/users/3759bdc5fdfb4c59b382c8c9748031ff/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:42.333036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bc611-fc5e-4806-8fac-bdaf995eb036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dbd55faf7a545119decc8f932782ae3, user_id=3759bdc5fdfb4c59b382c8c9748031ff, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:42.335751 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bc611-fc5e-4806-8fac-bdaf995eb036 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:42.431389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1000/2011] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:42 2022] PUT /identity/v3/projects/4dbd55faf7a545119decc8f932782ae3/users/3759bdc5fdfb4c59b382c8c9748031ff/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:42.443560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823fbec-87e5-4b7c-8f5c-dc4e53107250 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:42.443845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823fbec-87e5-4b7c-8f5c-dc4e53107250 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:42.444121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823fbec-87e5-4b7c-8f5c-dc4e53107250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:42.471615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8823fbec-87e5-4b7c-8f5c-dc4e53107250 None None] MFA Rules not processed for user `3759bdc5fdfb4c59b382c8c9748031ff`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:42.473232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8823fbec-87e5-4b7c-8f5c-dc4e53107250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:42.544821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1012/2012] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:42.549344 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bc611-fc5e-4806-8fac-bdaf995eb036 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:42.623036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bc611-fc5e-4806-8fac-bdaf995eb036 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:42.631306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d2482-85cb-4b5f-b541-3b76f32cf819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:42.632250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d2482-85cb-4b5f-b541-3b76f32cf819 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:42.632527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d2482-85cb-4b5f-b541-3b76f32cf819 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:42.632907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d2482-85cb-4b5f-b541-3b76f32cf819 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:42.633807 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d2482-85cb-4b5f-b541-3b76f32cf819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:42.636136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d2482-85cb-4b5f-b541-3b76f32cf819 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:42.713835 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:42.766018 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1001/2013] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:42 2022] POST /identity/v3/projects => generated 382 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:42.767051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823fbec-87e5-4b7c-8f5c-dc4e53107250 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:42.779177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823fbec-87e5-4b7c-8f5c-dc4e53107250 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:42.843735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb98be8-2cbe-4dbb-bc71-8df36e5dda0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:42.844418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb98be8-2cbe-4dbb-bc71-8df36e5dda0c admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:42.844955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb98be8-2cbe-4dbb-bc71-8df36e5dda0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:42.845368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb98be8-2cbe-4dbb-bc71-8df36e5dda0c admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:42.846080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb98be8-2cbe-4dbb-bc71-8df36e5dda0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:42.848517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb98be8-2cbe-4dbb-bc71-8df36e5dda0c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:42.974410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1013/2014] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:42 2022] POST /identity/v3/users => generated 298 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:42.978507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d2482-85cb-4b5f-b541-3b76f32cf819 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:42.989148 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d2482-85cb-4b5f-b541-3b76f32cf819 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:42.994104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f8f50-9db7-45ed-a8dc-d7418f0c2a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:42.994893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f8f50-9db7-45ed-a8dc-d7418f0c2a7c admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:42.995273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f8f50-9db7-45ed-a8dc-d7418f0c2a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:42.995621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f8f50-9db7-45ed-a8dc-d7418f0c2a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:42.996474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f8f50-9db7-45ed-a8dc-d7418f0c2a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:42.998152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f8f50-9db7-45ed-a8dc-d7418f0c2a7c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:43.026676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1002/2015] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:42 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:43.031297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb98be8-2cbe-4dbb-bc71-8df36e5dda0c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:43.044925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb98be8-2cbe-4dbb-bc71-8df36e5dda0c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:43.051738 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b25842-f731-460a-ae6c-2b3a9b6bdbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:43.052762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b25842-f731-460a-ae6c-2b3a9b6bdbf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:43.053183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b25842-f731-460a-ae6c-2b3a9b6bdbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:43.053548 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b25842-f731-460a-ae6c-2b3a9b6bdbf9 admin admin] PATH_INFO: `/v3/projects/f1af4db935fc403eaca44c67d90f4684/users/863e11685968447c8fbbfe762221ec6b/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:43.069718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b25842-f731-460a-ae6c-2b3a9b6bdbf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1af4db935fc403eaca44c67d90f4684, user_id=863e11685968447c8fbbfe762221ec6b, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:43.072456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b25842-f731-460a-ae6c-2b3a9b6bdbf9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:43.204488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1014/2016] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:43 2022] PUT /identity/v3/projects/f1af4db935fc403eaca44c67d90f4684/users/863e11685968447c8fbbfe762221ec6b/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:43.210473 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f8f50-9db7-45ed-a8dc-d7418f0c2a7c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:43.221694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f8f50-9db7-45ed-a8dc-d7418f0c2a7c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:43.301170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452a536-f7d6-4e94-9226-0d728b1e0461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:43.305530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452a536-f7d6-4e94-9226-0d728b1e0461 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:43.305530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452a536-f7d6-4e94-9226-0d728b1e0461 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:43.305530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452a536-f7d6-4e94-9226-0d728b1e0461 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:43.305530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452a536-f7d6-4e94-9226-0d728b1e0461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:43.305530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452a536-f7d6-4e94-9226-0d728b1e0461 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:43.316557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1003/2017] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:43 2022] GET /identity/v3/roles => generated 1427 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:43.321319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b25842-f731-460a-ae6c-2b3a9b6bdbf9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:43.332928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b25842-f731-460a-ae6c-2b3a9b6bdbf9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:43.339901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af73e0d-419c-418d-8fed-02c0879feaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:43.339901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af73e0d-419c-418d-8fed-02c0879feaa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:43.339901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af73e0d-419c-418d-8fed-02c0879feaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:43.339901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af73e0d-419c-418d-8fed-02c0879feaa4 admin admin] PATH_INFO: `/v3/projects/f1af4db935fc403eaca44c67d90f4684/users/863e11685968447c8fbbfe762221ec6b/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:43.341743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af73e0d-419c-418d-8fed-02c0879feaa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1af4db935fc403eaca44c67d90f4684, user_id=863e11685968447c8fbbfe762221ec6b, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:43.344483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af73e0d-419c-418d-8fed-02c0879feaa4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:43.357067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1015/2018] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:43 2022] PUT /identity/v3/projects/f1af4db935fc403eaca44c67d90f4684/users/863e11685968447c8fbbfe762221ec6b/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:43.368889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3dede-fefe-4318-812a-b5eae659e964 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:43.369182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3dede-fefe-4318-812a-b5eae659e964 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:43.369467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3dede-fefe-4318-812a-b5eae659e964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:43.402336 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-67b3dede-fefe-4318-812a-b5eae659e964 None None] MFA Rules not processed for user `863e11685968447c8fbbfe762221ec6b`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:43.403916 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-67b3dede-fefe-4318-812a-b5eae659e964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:43.477779 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1004/2019] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:43.491475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af73e0d-419c-418d-8fed-02c0879feaa4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:43.496746 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3af73e0d-419c-418d-8fed-02c0879feaa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:43.574346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af73e0d-419c-418d-8fed-02c0879feaa4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:43.578777 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a36970-01b5-4f10-ad8d-0e8811f00015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:43.579591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a36970-01b5-4f10-ad8d-0e8811f00015 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:43.579981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a36970-01b5-4f10-ad8d-0e8811f00015 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:43.580816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a36970-01b5-4f10-ad8d-0e8811f00015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:43.593350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a36970-01b5-4f10-ad8d-0e8811f00015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:43.594998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a36970-01b5-4f10-ad8d-0e8811f00015 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:43.607073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1016/2020] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:43.665577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:43.846352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:43.898448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3dede-fefe-4318-812a-b5eae659e964 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:43.908943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3dede-fefe-4318-812a-b5eae659e964 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:43.913781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f37f20-9fd9-4971-9de7-b1f76bd603e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:43.914702 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f37f20-9fd9-4971-9de7-b1f76bd603e3 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:43.915041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f37f20-9fd9-4971-9de7-b1f76bd603e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:43.915281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f37f20-9fd9-4971-9de7-b1f76bd603e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:43.983802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f37f20-9fd9-4971-9de7-b1f76bd603e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:43.985483 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f37f20-9fd9-4971-9de7-b1f76bd603e3 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:43.996972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1005/2021] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:44.963810 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a36970-01b5-4f10-ad8d-0e8811f00015 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:45.075002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a36970-01b5-4f10-ad8d-0e8811f00015 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:45.081334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:45.083364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:45.083364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:45.084086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] PATH_INFO: `/v3/users/3759bdc5fdfb4c59b382c8c9748031ff` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:45.085882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3759bdc5fdfb4c59b382c8c9748031ff)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:45.087708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:45.101458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:45.396662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3759bdc5fdfb4c59b382c8c9748031ff'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:45.405091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3759bdc5fdfb4c59b382c8c9748031ff'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:45.405991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3759bdc5fdfb4c59b382c8c9748031ff'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:45.426205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3759bdc5fdfb4c59b382c8c9748031ff'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:45.592681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3759bdc5fdfb4c59b382c8c9748031ff'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:45.601516 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1017/2022] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:44 2022] DELETE /identity/v3/users/3759bdc5fdfb4c59b382c8c9748031ff => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:45.605081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e3002-6512-40a2-828b-df1371e1f245 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:45.606255 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e3002-6512-40a2-828b-df1371e1f245 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:45.606636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e3002-6512-40a2-828b-df1371e1f245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:45.689919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:45.690969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-716e3002-6512-40a2-828b-df1371e1f245 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:45.693377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-716e3002-6512-40a2-828b-df1371e1f245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:45.770203 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1006/2023] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:45.781277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:45.783191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:45.954686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf5bc1-118f-48cb-ad55-ab6b60c901a5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:45.959042 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548160a0-61a4-4063-9c04-9015b489a98a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:45.959734 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548160a0-61a4-4063-9c04-9015b489a98a service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:45.960024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548160a0-61a4-4063-9c04-9015b489a98a service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:45.960326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548160a0-61a4-4063-9c04-9015b489a98a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:45.971019 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548160a0-61a4-4063-9c04-9015b489a98a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:45.973354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548160a0-61a4-4063-9c04-9015b489a98a service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:46.002996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1018/2024] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:46.491718 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e3002-6512-40a2-828b-df1371e1f245 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:46.494392 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-716e3002-6512-40a2-828b-df1371e1f245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:46.526970 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e3002-6512-40a2-828b-df1371e1f245 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:46.551346 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9d628-0f22-4077-beb2-486cd7db2c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:46.554715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9d628-0f22-4077-beb2-486cd7db2c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:46.555190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9d628-0f22-4077-beb2-486cd7db2c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:46.555658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9d628-0f22-4077-beb2-486cd7db2c73 admin admin] PATH_INFO: `/v3/users/cc27d0ab5aa44fe49c1e5a717b7a4d2d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:46.558140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9d628-0f22-4077-beb2-486cd7db2c73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc27d0ab5aa44fe49c1e5a717b7a4d2d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:46.565604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9d628-0f22-4077-beb2-486cd7db2c73 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:46.718272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:46.804263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548160a0-61a4-4063-9c04-9015b489a98a service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:46.805757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-548160a0-61a4-4063-9c04-9015b489a98a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:46.821136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548160a0-61a4-4063-9c04-9015b489a98a service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:46.826794 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:46.827730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:46.828111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:46.828279 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] PATH_INFO: `/v3/projects/4dbd55faf7a545119decc8f932782ae3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:46.829541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dbd55faf7a545119decc8f932782ae3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:46.836699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:46.852217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7be9d628-0f22-4077-beb2-486cd7db2c73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc27d0ab5aa44fe49c1e5a717b7a4d2d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:46.859375 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7be9d628-0f22-4077-beb2-486cd7db2c73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc27d0ab5aa44fe49c1e5a717b7a4d2d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:46.860194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7be9d628-0f22-4077-beb2-486cd7db2c73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc27d0ab5aa44fe49c1e5a717b7a4d2d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:46.872671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7be9d628-0f22-4077-beb2-486cd7db2c73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc27d0ab5aa44fe49c1e5a717b7a4d2d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.000489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7be9d628-0f22-4077-beb2-486cd7db2c73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc27d0ab5aa44fe49c1e5a717b7a4d2d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.007656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1007/2025] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:46 2022] DELETE /identity/v3/users/cc27d0ab5aa44fe49c1e5a717b7a4d2d => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:47.013484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a394b33-1fde-4c43-b6c7-14237c5f28bb None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:47.013870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a394b33-1fde-4c43-b6c7-14237c5f28bb None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:47.014157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a394b33-1fde-4c43-b6c7-14237c5f28bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:47.022052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dbd55faf7a545119decc8f932782ae3', 'request_id': 'req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.039615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-6a394b33-1fde-4c43-b6c7-14237c5f28bb None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:47.040804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6a394b33-1fde-4c43-b6c7-14237c5f28bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:47.108407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1008/2026] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:47.118261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a394b33-1fde-4c43-b6c7-14237c5f28bb None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:47.119737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6a394b33-1fde-4c43-b6c7-14237c5f28bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:47.124124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dbd55faf7a545119decc8f932782ae3', 'request_id': 'req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.131858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dbd55faf7a545119decc8f932782ae3', 'request_id': 'req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.145269 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] The token cache is being invalidate because project 4dbd55faf7a545119decc8f932782ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:47.145459 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.147164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.149073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1019/2027] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:46 2022] DELETE /identity/v3/projects/4dbd55faf7a545119decc8f932782ae3 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:47.152845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:47.154829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:47.190030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a394b33-1fde-4c43-b6c7-14237c5f28bb None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:47.193974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff120e04-8cbc-43a5-9c04-af7b0d5048c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:47.194700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff120e04-8cbc-43a5-9c04-af7b0d5048c5 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:47.194970 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff120e04-8cbc-43a5-9c04-af7b0d5048c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:47.199907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff120e04-8cbc-43a5-9c04-af7b0d5048c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:47.199907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ff120e04-8cbc-43a5-9c04-af7b0d5048c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:47.211403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff120e04-8cbc-43a5-9c04-af7b0d5048c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:47.213143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff120e04-8cbc-43a5-9c04-af7b0d5048c5 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:47.231287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1009/2028] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:47.238920 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167115d-a03c-4c0d-9a11-f18c9c15d3e9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:47.243999 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:47.244899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:47.245183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:47.245460 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] PATH_INFO: `/v3/users/863e11685968447c8fbbfe762221ec6b` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:47.247087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=863e11685968447c8fbbfe762221ec6b)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:47.249551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:47.499896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff120e04-8cbc-43a5-9c04-af7b0d5048c5 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:47.502501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ff120e04-8cbc-43a5-9c04-af7b0d5048c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:47.520632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff120e04-8cbc-43a5-9c04-af7b0d5048c5 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:47.532607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a0c63-4c64-4f22-b621-54a93fa1b7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:47.532607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a0c63-4c64-4f22-b621-54a93fa1b7f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:47.532607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a0c63-4c64-4f22-b621-54a93fa1b7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:47.532607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a0c63-4c64-4f22-b621-54a93fa1b7f9 admin admin] PATH_INFO: `/v3/users/f232a5374b71455b8efb4f5e3bf593d7` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:47.533878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a0c63-4c64-4f22-b621-54a93fa1b7f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f232a5374b71455b8efb4f5e3bf593d7)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:47.534541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a0c63-4c64-4f22-b621-54a93fa1b7f9 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:47.577877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '863e11685968447c8fbbfe762221ec6b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.592742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '863e11685968447c8fbbfe762221ec6b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.593736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '863e11685968447c8fbbfe762221ec6b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.610593 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '863e11685968447c8fbbfe762221ec6b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.722404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '863e11685968447c8fbbfe762221ec6b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.730321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1020/2029] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:47 2022] DELETE /identity/v3/users/863e11685968447c8fbbfe762221ec6b => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:47.784581 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1e1a0c63-4c64-4f22-b621-54a93fa1b7f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f232a5374b71455b8efb4f5e3bf593d7'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.793021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1e1a0c63-4c64-4f22-b621-54a93fa1b7f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f232a5374b71455b8efb4f5e3bf593d7'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.795945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1e1a0c63-4c64-4f22-b621-54a93fa1b7f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f232a5374b71455b8efb4f5e3bf593d7'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.808403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1e1a0c63-4c64-4f22-b621-54a93fa1b7f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f232a5374b71455b8efb4f5e3bf593d7'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.833869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:47.837084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:47.921769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1e1a0c63-4c64-4f22-b621-54a93fa1b7f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f232a5374b71455b8efb4f5e3bf593d7'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:47.923316 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a914ab-8fe9-44f4-926c-5b0df6c00e63 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:47.929996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:47.930222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1010/2030] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:47 2022] DELETE /identity/v3/users/f232a5374b71455b8efb4f5e3bf593d7 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:47.930941 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:47.931224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:47.931552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] PATH_INFO: `/v3/projects/ad64221d33dd460b93aec895d46c8765` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:47.932772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad64221d33dd460b93aec895d46c8765)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:47.934797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:47.935179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad029-0201-4a00-82cc-f8bca22826bb None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:47.935861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad029-0201-4a00-82cc-f8bca22826bb None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:47.936359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad029-0201-4a00-82cc-f8bca22826bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:47.960889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-7b0ad029-0201-4a00-82cc-f8bca22826bb None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:47.962381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7b0ad029-0201-4a00-82cc-f8bca22826bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:48.023791 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1011/2031] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:48.033984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad029-0201-4a00-82cc-f8bca22826bb None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:48.038641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7b0ad029-0201-4a00-82cc-f8bca22826bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:48.053826 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad64221d33dd460b93aec895d46c8765', 'request_id': 'req-a5005cff-a2c4-47f2-aff5-67d5afab1117'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.109028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad029-0201-4a00-82cc-f8bca22826bb None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:48.138546 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad64221d33dd460b93aec895d46c8765', 'request_id': 'req-a5005cff-a2c4-47f2-aff5-67d5afab1117'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.147635 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad64221d33dd460b93aec895d46c8765', 'request_id': 'req-a5005cff-a2c4-47f2-aff5-67d5afab1117'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.171353 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] The token cache is being invalidate because project ad64221d33dd460b93aec895d46c8765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:48.171620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.173595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.177623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1021/2032] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:47 2022] DELETE /identity/v3/projects/ad64221d33dd460b93aec895d46c8765 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:48.180994 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:48.183671 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:48.185320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:48.187495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6dbda-4e3a-4113-8144-d3cbc87de260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:48.188193 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6dbda-4e3a-4113-8144-d3cbc87de260 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:48.188471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6dbda-4e3a-4113-8144-d3cbc87de260 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:48.188805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6dbda-4e3a-4113-8144-d3cbc87de260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:48.190783 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d6c6dbda-4e3a-4113-8144-d3cbc87de260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:48.264399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6dbda-4e3a-4113-8144-d3cbc87de260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:48.265978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6dbda-4e3a-4113-8144-d3cbc87de260 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:48.277064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1012/2033] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:48.287544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5005cff-a2c4-47f2-aff5-67d5afab1117 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:48.294855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:48.296133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:48.296423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:48.296688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] PATH_INFO: `/v3/users/c4ed42b1681e4a5b8c1afe867701c77a` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:48.297883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4ed42b1681e4a5b8c1afe867701c77a)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:48.299942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:48.324677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6dbda-4e3a-4113-8144-d3cbc87de260 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:48.326177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d6c6dbda-4e3a-4113-8144-d3cbc87de260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:48.340503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6dbda-4e3a-4113-8144-d3cbc87de260 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:48.347068 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a4e99-80e4-48f1-87d8-c11e8edaa8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:48.349377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a4e99-80e4-48f1-87d8-c11e8edaa8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:48.349663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a4e99-80e4-48f1-87d8-c11e8edaa8be admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:48.349937 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a4e99-80e4-48f1-87d8-c11e8edaa8be admin admin] PATH_INFO: `/v3/projects/f1af4db935fc403eaca44c67d90f4684` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:48.352301 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a4e99-80e4-48f1-87d8-c11e8edaa8be admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1af4db935fc403eaca44c67d90f4684)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:48.354489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a4e99-80e4-48f1-87d8-c11e8edaa8be admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:48.669942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-633a4e99-80e4-48f1-87d8-c11e8edaa8be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1af4db935fc403eaca44c67d90f4684', 'request_id': 'req-633a4e99-80e4-48f1-87d8-c11e8edaa8be'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.671525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4ed42b1681e4a5b8c1afe867701c77a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.684311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4ed42b1681e4a5b8c1afe867701c77a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.685118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4ed42b1681e4a5b8c1afe867701c77a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.703417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4ed42b1681e4a5b8c1afe867701c77a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.752201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-633a4e99-80e4-48f1-87d8-c11e8edaa8be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1af4db935fc403eaca44c67d90f4684', 'request_id': 'req-633a4e99-80e4-48f1-87d8-c11e8edaa8be'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.759461 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-633a4e99-80e4-48f1-87d8-c11e8edaa8be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1af4db935fc403eaca44c67d90f4684', 'request_id': 'req-633a4e99-80e4-48f1-87d8-c11e8edaa8be'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.771190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-633a4e99-80e4-48f1-87d8-c11e8edaa8be admin admin] The token cache is being invalidate because project f1af4db935fc403eaca44c67d90f4684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:48.771473 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-633a4e99-80e4-48f1-87d8-c11e8edaa8be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.772299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-633a4e99-80e4-48f1-87d8-c11e8edaa8be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.773854 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1013/2034] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:48 2022] DELETE /identity/v3/projects/f1af4db935fc403eaca44c67d90f4684 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:48.780218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d490b7-bc8f-4d52-b43f-c256c151fe44 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:48.780802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d490b7-bc8f-4d52-b43f-c256c151fe44 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:48.781261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d490b7-bc8f-4d52-b43f-c256c151fe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:48.829237 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b5d490b7-bc8f-4d52-b43f-c256c151fe44 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:48.830790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b5d490b7-bc8f-4d52-b43f-c256c151fe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:48.839462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4ed42b1681e4a5b8c1afe867701c77a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:48.852760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1022/2035] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:48 2022] DELETE /identity/v3/users/c4ed42b1681e4a5b8c1afe867701c77a => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:48.857891 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:48.863914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:48.941932 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1014/2036] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:48.952150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1c9ed-3ba1-453e-8b62-dcee0b9a714a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:48.952447 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1c9ed-3ba1-453e-8b62-dcee0b9a714a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:48.952721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1c9ed-3ba1-453e-8b62-dcee0b9a714a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:48.979585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-81f1c9ed-3ba1-453e-8b62-dcee0b9a714a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:48.981580 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-81f1c9ed-3ba1-453e-8b62-dcee0b9a714a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:48.985721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ce8e9-32a4-4fba-b43c-08928ae1439e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:48.994091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:48.996767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:48.997107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:48.997413 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] PATH_INFO: `/v3/projects/f81fb10777f541178958502badde88bc` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:49.002137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1015/2037] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:49.002313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f81fb10777f541178958502badde88bc)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:49.002313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:49.005269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1c9ed-3ba1-453e-8b62-dcee0b9a714a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:49.015766 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1c9ed-3ba1-453e-8b62-dcee0b9a714a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:49.031408 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb61edd-1cf7-4c95-84c5-2a8d9222a93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:49.032105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb61edd-1cf7-4c95-84c5-2a8d9222a93d admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:49.034602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb61edd-1cf7-4c95-84c5-2a8d9222a93d admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:49.034602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb61edd-1cf7-4c95-84c5-2a8d9222a93d admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:49.034602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb61edd-1cf7-4c95-84c5-2a8d9222a93d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:49.034602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb61edd-1cf7-4c95-84c5-2a8d9222a93d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:49.035999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb61edd-1cf7-4c95-84c5-2a8d9222a93d admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:49.070925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1016/2038] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:49.078751 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb61edd-1cf7-4c95-84c5-2a8d9222a93d admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:49.090673 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb61edd-1cf7-4c95-84c5-2a8d9222a93d admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:49.100237 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0e436-8c66-4346-ae98-4c113f510f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:49.101744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0e436-8c66-4346-ae98-4c113f510f2c admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:49.103053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0e436-8c66-4346-ae98-4c113f510f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:49.109485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0e436-8c66-4346-ae98-4c113f510f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:49.109485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0e436-8c66-4346-ae98-4c113f510f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:49.109485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0e436-8c66-4346-ae98-4c113f510f2c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:49.163246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f81fb10777f541178958502badde88bc', 'request_id': 'req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:49.256851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f81fb10777f541178958502badde88bc', 'request_id': 'req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:49.265660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f81fb10777f541178958502badde88bc', 'request_id': 'req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:49.266807 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1017/2039] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:49 2022] POST /identity/v3/projects => generated 376 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:49.267711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0e436-8c66-4346-ae98-4c113f510f2c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:49.283050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] The token cache is being invalidate because project f81fb10777f541178958502badde88bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:49.283510 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:49.298169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:49.305159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1023/2040] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:48 2022] DELETE /identity/v3/projects/f81fb10777f541178958502badde88bc => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:49.308379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:49.310238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:49.369031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0e436-8c66-4346-ae98-4c113f510f2c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:49.375282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bc80b-c7e2-4e9f-b3b0-33da4b28a139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:49.376271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bc80b-c7e2-4e9f-b3b0-33da4b28a139 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:49.377028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bc80b-c7e2-4e9f-b3b0-33da4b28a139 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:49.377028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bc80b-c7e2-4e9f-b3b0-33da4b28a139 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:49.377536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bc80b-c7e2-4e9f-b3b0-33da4b28a139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:49.380588 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bc80b-c7e2-4e9f-b3b0-33da4b28a139 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:49.417519 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ed15d-3c63-4dc6-a4b5-aace39e4b2d2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:49.431386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:49.432587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:49.433025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:49.433522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] PATH_INFO: `/v3/users/a826f366ec57437eb50bd628d09801a6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:49.441273 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a826f366ec57437eb50bd628d09801a6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:49.443419 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:49.524679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1018/2041] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:49 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:49.529093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bc80b-c7e2-4e9f-b3b0-33da4b28a139 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:49.531117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b43bc80b-c7e2-4e9f-b3b0-33da4b28a139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:49.543175 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:49.548188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bc80b-c7e2-4e9f-b3b0-33da4b28a139 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:49.553308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7e579-e079-4d3e-b63f-9650b3338bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:49.554099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7e579-e079-4d3e-b63f-9650b3338bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:49.554516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7e579-e079-4d3e-b63f-9650b3338bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:49.554920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7e579-e079-4d3e-b63f-9650b3338bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:49.555800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7e579-e079-4d3e-b63f-9650b3338bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:49.557972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7e579-e079-4d3e-b63f-9650b3338bd7 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:49.570495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1019/2042] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:49 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:49.574643 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7e579-e079-4d3e-b63f-9650b3338bd7 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:49.592227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7e579-e079-4d3e-b63f-9650b3338bd7 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:49.605996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784b64e-6ba0-441c-8f51-80ef39ec69ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:49.606768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784b64e-6ba0-441c-8f51-80ef39ec69ab admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:49.612046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784b64e-6ba0-441c-8f51-80ef39ec69ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:49.614541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784b64e-6ba0-441c-8f51-80ef39ec69ab admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:49.614541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784b64e-6ba0-441c-8f51-80ef39ec69ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:49.619277 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784b64e-6ba0-441c-8f51-80ef39ec69ab admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:49.655524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1020/2043] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:49 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:49.656804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784b64e-6ba0-441c-8f51-80ef39ec69ab admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:49.659845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b784b64e-6ba0-441c-8f51-80ef39ec69ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:49.687875 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784b64e-6ba0-441c-8f51-80ef39ec69ab admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:49.694668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae8f8c-d799-4912-b142-3a8bf961ef5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:49.696249 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae8f8c-d799-4912-b142-3a8bf961ef5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:49.696690 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae8f8c-d799-4912-b142-3a8bf961ef5a admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:49.697218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae8f8c-d799-4912-b142-3a8bf961ef5a admin admin] PATH_INFO: `/v3/projects/20bbd9ef0f234d3da52d2445122237b4` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:49.698958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae8f8c-d799-4912-b142-3a8bf961ef5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=20bbd9ef0f234d3da52d2445122237b4)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:49.701612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae8f8c-d799-4912-b142-3a8bf961ef5a admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:49.801966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a826f366ec57437eb50bd628d09801a6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:49.817080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a826f366ec57437eb50bd628d09801a6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:49.818027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a826f366ec57437eb50bd628d09801a6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:49.831948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a826f366ec57437eb50bd628d09801a6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:49.845063 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-02ae8f8c-d799-4912-b142-3a8bf961ef5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20bbd9ef0f234d3da52d2445122237b4', 'request_id': 'req-02ae8f8c-d799-4912-b142-3a8bf961ef5a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:49.929919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a826f366ec57437eb50bd628d09801a6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:49.938593 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1024/2044] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:49 2022] DELETE /identity/v3/users/a826f366ec57437eb50bd628d09801a6 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:49.943797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:49.949633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:50.007336 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-02ae8f8c-d799-4912-b142-3a8bf961ef5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20bbd9ef0f234d3da52d2445122237b4', 'request_id': 'req-02ae8f8c-d799-4912-b142-3a8bf961ef5a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:50.016949 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-02ae8f8c-d799-4912-b142-3a8bf961ef5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20bbd9ef0f234d3da52d2445122237b4', 'request_id': 'req-02ae8f8c-d799-4912-b142-3a8bf961ef5a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:50.038003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-02ae8f8c-d799-4912-b142-3a8bf961ef5a admin admin] The token cache is being invalidate because project 20bbd9ef0f234d3da52d2445122237b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:50.038339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-02ae8f8c-d799-4912-b142-3a8bf961ef5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:50.039188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-02ae8f8c-d799-4912-b142-3a8bf961ef5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:50.047841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1021/2045] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:49 2022] DELETE /identity/v3/projects/20bbd9ef0f234d3da52d2445122237b4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:50.053951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9b4eb-f27f-431e-9c6a-ad72f88f7814 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.054504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9b4eb-f27f-431e-9c6a-ad72f88f7814 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.064946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9b4eb-f27f-431e-9c6a-ad72f88f7814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.094331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3934c88-b421-4c94-b844-536c7c17645e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.101517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808a1d4-9b6e-49f0-907e-f5a57b19f9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.104082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808a1d4-9b6e-49f0-907e-f5a57b19f9fc admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.105661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808a1d4-9b6e-49f0-907e-f5a57b19f9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.106331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808a1d4-9b6e-49f0-907e-f5a57b19f9fc admin admin] PATH_INFO: `/v3/projects/afe657943fa44b88baffcf9fd273b4e9/users/556aab8b6cb04d06bbf49250a456aaf0/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.127097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-84d9b4eb-f27f-431e-9c6a-ad72f88f7814 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:50.128346 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-84d9b4eb-f27f-431e-9c6a-ad72f88f7814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:50.133627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808a1d4-9b6e-49f0-907e-f5a57b19f9fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe657943fa44b88baffcf9fd273b4e9, user_id=556aab8b6cb04d06bbf49250a456aaf0, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.149601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808a1d4-9b6e-49f0-907e-f5a57b19f9fc admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.188749 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1022/2046] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:50.198369 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ad028-3442-4bea-803e-10a91ead27f4 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.198660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ad028-3442-4bea-803e-10a91ead27f4 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.199006 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ad028-3442-4bea-803e-10a91ead27f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.253698 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-278ad028-3442-4bea-803e-10a91ead27f4 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:50.272252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-278ad028-3442-4bea-803e-10a91ead27f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:50.294112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1023/2047] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:50.297972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ad028-3442-4bea-803e-10a91ead27f4 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.302049 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1025/2048] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:49 2022] PUT /identity/v3/projects/afe657943fa44b88baffcf9fd273b4e9/users/556aab8b6cb04d06bbf49250a456aaf0/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:50.315028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b0e07-f05d-4bb2-9231-299bc906d68c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.317330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b0e07-f05d-4bb2-9231-299bc906d68c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.317824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b0e07-f05d-4bb2-9231-299bc906d68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.321439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ad028-3442-4bea-803e-10a91ead27f4 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.354294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f61b0e07-f05d-4bb2-9231-299bc906d68c None None] MFA Rules not processed for user `556aab8b6cb04d06bbf49250a456aaf0`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:50.356131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f61b0e07-f05d-4bb2-9231-299bc906d68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:50.413898 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fad3f-b90a-46ec-95b5-eb7b033622c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.414621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fad3f-b90a-46ec-95b5-eb7b033622c5 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.414933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fad3f-b90a-46ec-95b5-eb7b033622c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.415211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fad3f-b90a-46ec-95b5-eb7b033622c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.415976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fad3f-b90a-46ec-95b5-eb7b033622c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:50.426986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fad3f-b90a-46ec-95b5-eb7b033622c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.430325 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fad3f-b90a-46ec-95b5-eb7b033622c5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.445303 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1024/2049] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:00:50.446929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fad3f-b90a-46ec-95b5-eb7b033622c5 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.458717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fad3f-b90a-46ec-95b5-eb7b033622c5 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.464329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83882dbc-85c0-4ec1-8ed5-b154c0cd7e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.465026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83882dbc-85c0-4ec1-8ed5-b154c0cd7e7f admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.465305 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83882dbc-85c0-4ec1-8ed5-b154c0cd7e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.465591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83882dbc-85c0-4ec1-8ed5-b154c0cd7e7f admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.473156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83882dbc-85c0-4ec1-8ed5-b154c0cd7e7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.476973 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83882dbc-85c0-4ec1-8ed5-b154c0cd7e7f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.479805 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1026/2050] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:50.494930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b0e07-f05d-4bb2-9231-299bc906d68c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.496538 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f61b0e07-f05d-4bb2-9231-299bc906d68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:50.608880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1025/2051] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:50 2022] POST /identity/v3/projects => generated 376 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:50.611903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83882dbc-85c0-4ec1-8ed5-b154c0cd7e7f admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.667176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b0e07-f05d-4bb2-9231-299bc906d68c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.725142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83882dbc-85c0-4ec1-8ed5-b154c0cd7e7f admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.732705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056480e9-367e-406f-8f50-d2b511b8571f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.733431 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056480e9-367e-406f-8f50-d2b511b8571f admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.733716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056480e9-367e-406f-8f50-d2b511b8571f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.734023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056480e9-367e-406f-8f50-d2b511b8571f admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.734759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056480e9-367e-406f-8f50-d2b511b8571f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.737003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056480e9-367e-406f-8f50-d2b511b8571f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.739136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df062cc-2617-42a1-b6c8-d49beb92c16f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.739982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df062cc-2617-42a1-b6c8-d49beb92c16f service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.740310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df062cc-2617-42a1-b6c8-d49beb92c16f service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.740532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df062cc-2617-42a1-b6c8-d49beb92c16f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.834950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df062cc-2617-42a1-b6c8-d49beb92c16f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.836551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df062cc-2617-42a1-b6c8-d49beb92c16f service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.862279 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1027/2052] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:50.864868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df062cc-2617-42a1-b6c8-d49beb92c16f service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.867156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7df062cc-2617-42a1-b6c8-d49beb92c16f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:50.885010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df062cc-2617-42a1-b6c8-d49beb92c16f service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.890939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e077fb-726e-4364-a96d-6d7f7aaaecc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.892182 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e077fb-726e-4364-a96d-6d7f7aaaecc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.892567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e077fb-726e-4364-a96d-6d7f7aaaecc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.892994 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e077fb-726e-4364-a96d-6d7f7aaaecc6 admin admin] PATH_INFO: `/v3/projects/cb70021a2fd94717840fa3fd00a7a7d0` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.893510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1026/2053] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:50 2022] POST /identity/v3/users => generated 289 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:50.894395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e077fb-726e-4364-a96d-6d7f7aaaecc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb70021a2fd94717840fa3fd00a7a7d0)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.896567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e077fb-726e-4364-a96d-6d7f7aaaecc6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.897421 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056480e9-367e-406f-8f50-d2b511b8571f admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.933502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056480e9-367e-406f-8f50-d2b511b8571f admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.941062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68bf2e-0883-4f3a-a70b-26bd45ec8243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.943623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68bf2e-0883-4f3a-a70b-26bd45ec8243 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.943623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68bf2e-0883-4f3a-a70b-26bd45ec8243 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.943623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68bf2e-0883-4f3a-a70b-26bd45ec8243 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.943935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68bf2e-0883-4f3a-a70b-26bd45ec8243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.945335 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68bf2e-0883-4f3a-a70b-26bd45ec8243 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.964490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1027/2054] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:50 2022] GET /identity/v3/roles => generated 1427 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:50.969762 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68bf2e-0883-4f3a-a70b-26bd45ec8243 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.980516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68bf2e-0883-4f3a-a70b-26bd45ec8243 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.989116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b47d6-aad9-4d17-baf7-f7d2c5faf555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.990104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b47d6-aad9-4d17-baf7-f7d2c5faf555 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.990534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b47d6-aad9-4d17-baf7-f7d2c5faf555 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.991064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b47d6-aad9-4d17-baf7-f7d2c5faf555 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.992359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b47d6-aad9-4d17-baf7-f7d2c5faf555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:51.005450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b47d6-aad9-4d17-baf7-f7d2c5faf555 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:51.031792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1028/2055] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:50 2022] GET /identity/v3/roles => generated 1427 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:51.039050 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b47d6-aad9-4d17-baf7-f7d2c5faf555 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:51.059602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b47d6-aad9-4d17-baf7-f7d2c5faf555 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:51.064405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-73e077fb-726e-4364-a96d-6d7f7aaaecc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb70021a2fd94717840fa3fd00a7a7d0', 'request_id': 'req-73e077fb-726e-4364-a96d-6d7f7aaaecc6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:51.135973 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5ef4f-fba7-4984-9004-78a5e52c31e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:51.136948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5ef4f-fba7-4984-9004-78a5e52c31e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.137854 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5ef4f-fba7-4984-9004-78a5e52c31e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.138145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5ef4f-fba7-4984-9004-78a5e52c31e0 admin admin] PATH_INFO: `/v3/projects/a7cf21850a5d4df7b7e3d145c0c2402b/users/d22ffa619cbd426381c525736af882f6/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.156155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5ef4f-fba7-4984-9004-78a5e52c31e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7cf21850a5d4df7b7e3d145c0c2402b, user_id=d22ffa619cbd426381c525736af882f6, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:51.162902 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5ef4f-fba7-4984-9004-78a5e52c31e0 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:51.184708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-73e077fb-726e-4364-a96d-6d7f7aaaecc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb70021a2fd94717840fa3fd00a7a7d0', 'request_id': 'req-73e077fb-726e-4364-a96d-6d7f7aaaecc6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:51.187374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:51.199550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-73e077fb-726e-4364-a96d-6d7f7aaaecc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb70021a2fd94717840fa3fd00a7a7d0', 'request_id': 'req-73e077fb-726e-4364-a96d-6d7f7aaaecc6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:51.212523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-73e077fb-726e-4364-a96d-6d7f7aaaecc6 admin admin] The token cache is being invalidate because project cb70021a2fd94717840fa3fd00a7a7d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:51.213044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-73e077fb-726e-4364-a96d-6d7f7aaaecc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:51.213907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-73e077fb-726e-4364-a96d-6d7f7aaaecc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:51.216251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1028/2056] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:50 2022] DELETE /identity/v3/projects/cb70021a2fd94717840fa3fd00a7a7d0 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:51.220993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58050923-367b-41af-8516-0fe2dbbf006e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.221278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58050923-367b-41af-8516-0fe2dbbf006e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.221638 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58050923-367b-41af-8516-0fe2dbbf006e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.241705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-58050923-367b-41af-8516-0fe2dbbf006e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:51.243068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-58050923-367b-41af-8516-0fe2dbbf006e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:51.255641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1029/2057] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:51 2022] PUT /identity/v3/projects/a7cf21850a5d4df7b7e3d145c0c2402b/users/d22ffa619cbd426381c525736af882f6/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:51.265388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281c26a-f185-4022-a546-dc49685679a3 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.265862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281c26a-f185-4022-a546-dc49685679a3 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.266243 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281c26a-f185-4022-a546-dc49685679a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.285374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1029/2058] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:51.287202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838063ef-631b-41fc-b89b-4ad97099278d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.287485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838063ef-631b-41fc-b89b-4ad97099278d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.287755 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838063ef-631b-41fc-b89b-4ad97099278d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.314028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1281c26a-f185-4022-a546-dc49685679a3 None None] MFA Rules not processed for user `d22ffa619cbd426381c525736af882f6`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:51.315388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1281c26a-f185-4022-a546-dc49685679a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:51.325562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-838063ef-631b-41fc-b89b-4ad97099278d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:51.327006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-838063ef-631b-41fc-b89b-4ad97099278d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:51.399473 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1030/2059] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:51.402073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1030/2060] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:51.407036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281c26a-f185-4022-a546-dc49685679a3 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:51.412960 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838063ef-631b-41fc-b89b-4ad97099278d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:51.414339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-838063ef-631b-41fc-b89b-4ad97099278d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:51.419979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281c26a-f185-4022-a546-dc49685679a3 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:51.425568 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49daa87-69a6-459e-82de-15a5fdf41f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:51.426499 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49daa87-69a6-459e-82de-15a5fdf41f2b admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.426906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49daa87-69a6-459e-82de-15a5fdf41f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.427451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49daa87-69a6-459e-82de-15a5fdf41f2b admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.428300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49daa87-69a6-459e-82de-15a5fdf41f2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:51.428622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49daa87-69a6-459e-82de-15a5fdf41f2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:51.430333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49daa87-69a6-459e-82de-15a5fdf41f2b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:51.445289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1031/2061] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:51.450050 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49daa87-69a6-459e-82de-15a5fdf41f2b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:51.457801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:51.465841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49daa87-69a6-459e-82de-15a5fdf41f2b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:51.475074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7ef01-a112-45d4-a8c7-331611308018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:51.475883 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7ef01-a112-45d4-a8c7-331611308018 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.476167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7ef01-a112-45d4-a8c7-331611308018 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.476661 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7ef01-a112-45d4-a8c7-331611308018 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.477658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7ef01-a112-45d4-a8c7-331611308018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:51.480710 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7ef01-a112-45d4-a8c7-331611308018 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:51.487464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838063ef-631b-41fc-b89b-4ad97099278d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:51.492178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d21b68-142a-4ac4-bef8-83829914c465 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '683c54f489f440b9842cb94980214df1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:51.492964 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d21b68-142a-4ac4-bef8-83829914c465 service glance] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.493251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d21b68-142a-4ac4-bef8-83829914c465 service glance] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.493666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d21b68-142a-4ac4-bef8-83829914c465 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.503563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d21b68-142a-4ac4-bef8-83829914c465 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:51.505581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d21b68-142a-4ac4-bef8-83829914c465 service glance] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:51.516321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1031/2062] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:51.598675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1032/2063] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:51 2022] POST /identity/v3/projects => generated 368 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:51.602691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d21b68-142a-4ac4-bef8-83829914c465 service glance] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:51.614040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d21b68-142a-4ac4-bef8-83829914c465 service glance] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:51.706262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1b5a9-f888-4f2e-81e2-7e276fcf481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:51.707690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1b5a9-f888-4f2e-81e2-7e276fcf481f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.708065 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1b5a9-f888-4f2e-81e2-7e276fcf481f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.708346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1b5a9-f888-4f2e-81e2-7e276fcf481f admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.709146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1b5a9-f888-4f2e-81e2-7e276fcf481f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:51.711587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1b5a9-f888-4f2e-81e2-7e276fcf481f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:51.815797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1032/2064] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:51 2022] POST /identity/v3/users => generated 285 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:51.817850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7ef01-a112-45d4-a8c7-331611308018 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:51.849381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7ef01-a112-45d4-a8c7-331611308018 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:51.862876 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e3df5-2940-415a-bd26-728c1ba84add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:51.865742 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e3df5-2940-415a-bd26-728c1ba84add admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.871099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e3df5-2940-415a-bd26-728c1ba84add admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.872155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e3df5-2940-415a-bd26-728c1ba84add admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.873443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e3df5-2940-415a-bd26-728c1ba84add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:51.875670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e3df5-2940-415a-bd26-728c1ba84add admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:51.894099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1033/2065] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:51 2022] GET /identity/v3/roles => generated 1427 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:51.895006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1b5a9-f888-4f2e-81e2-7e276fcf481f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:51.909702 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1b5a9-f888-4f2e-81e2-7e276fcf481f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:51.916497 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528a4ac-eb32-43c7-8d2c-fe53d4789fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:51.917163 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528a4ac-eb32-43c7-8d2c-fe53d4789fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.917446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528a4ac-eb32-43c7-8d2c-fe53d4789fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.917721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528a4ac-eb32-43c7-8d2c-fe53d4789fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.918454 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528a4ac-eb32-43c7-8d2c-fe53d4789fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:51.920220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528a4ac-eb32-43c7-8d2c-fe53d4789fc7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:51.934859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1033/2066] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:51 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:51.938458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e3df5-2940-415a-bd26-728c1ba84add admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:51.951318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e3df5-2940-415a-bd26-728c1ba84add admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:51.960821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e5c26-85a8-45a1-b840-3738433568c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:51.960821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e5c26-85a8-45a1-b840-3738433568c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.960821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e5c26-85a8-45a1-b840-3738433568c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.960821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e5c26-85a8-45a1-b840-3738433568c2 admin admin] PATH_INFO: `/v3/projects/f5ad3445821c4925900089667a19f58c/users/d0bdd63e8e0f466f8f38ce6fd0bb2d29/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.981227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e5c26-85a8-45a1-b840-3738433568c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ad3445821c4925900089667a19f58c, user_id=d0bdd63e8e0f466f8f38ce6fd0bb2d29, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:51.983941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e5c26-85a8-45a1-b840-3738433568c2 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:52.122205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1034/2067] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:51 2022] PUT /identity/v3/projects/f5ad3445821c4925900089667a19f58c/users/d0bdd63e8e0f466f8f38ce6fd0bb2d29/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:52.132791 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e140519-7b7f-40a8-a7b4-f441b709a2e3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:52.132933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e140519-7b7f-40a8-a7b4-f441b709a2e3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:52.133210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e140519-7b7f-40a8-a7b4-f441b709a2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:52.164314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1e140519-7b7f-40a8-a7b4-f441b709a2e3 None None] MFA Rules not processed for user `d0bdd63e8e0f466f8f38ce6fd0bb2d29`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:52.165976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1e140519-7b7f-40a8-a7b4-f441b709a2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:52.231090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1034/2068] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:52.235385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e5c26-85a8-45a1-b840-3738433568c2 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:52.307922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e5c26-85a8-45a1-b840-3738433568c2 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:52.314179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561e2a4-4eb2-485a-999d-6fe707f18927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:52.315216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561e2a4-4eb2-485a-999d-6fe707f18927 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:52.315408 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561e2a4-4eb2-485a-999d-6fe707f18927 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:52.315669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561e2a4-4eb2-485a-999d-6fe707f18927 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:52.316419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561e2a4-4eb2-485a-999d-6fe707f18927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:52.318350 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561e2a4-4eb2-485a-999d-6fe707f18927 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:52.424756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1035/2069] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:52 2022] POST /identity/v3/projects => generated 368 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:52.426666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e140519-7b7f-40a8-a7b4-f441b709a2e3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:52.438408 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e140519-7b7f-40a8-a7b4-f441b709a2e3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:52.509890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dff5af-ffe6-4f13-854a-f7557ed78fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:52.510574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dff5af-ffe6-4f13-854a-f7557ed78fcc admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:52.510883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dff5af-ffe6-4f13-854a-f7557ed78fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:52.511372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dff5af-ffe6-4f13-854a-f7557ed78fcc admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:52.512081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dff5af-ffe6-4f13-854a-f7557ed78fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:52.514079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dff5af-ffe6-4f13-854a-f7557ed78fcc admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:52.653257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1035/2070] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:52 2022] POST /identity/v3/users => generated 291 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:52.655463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f02092-d980-4499-aa2a-d3a77da69587 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:52.655806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f02092-d980-4499-aa2a-d3a77da69587 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:52.656211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f02092-d980-4499-aa2a-d3a77da69587 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:52.657632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1036/2071] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 18:00:52 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) May 26 18:00:52.658708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dff5af-ffe6-4f13-854a-f7557ed78fcc admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:52.669534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5109d35-341c-4dca-a506-daaee6f57e31 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:52.670065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5109d35-341c-4dca-a506-daaee6f57e31 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:52.670273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5109d35-341c-4dca-a506-daaee6f57e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:52.677168 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dff5af-ffe6-4f13-854a-f7557ed78fcc admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:52.693575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f8f00-e3a5-4447-b4a0-a03e343af553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:52.694447 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b5109d35-341c-4dca-a506-daaee6f57e31 None None] MFA Rules not processed for user `7847d2c119474d49bef6029d51dbefdb`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:52.694814 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f8f00-e3a5-4447-b4a0-a03e343af553 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:52.695371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f8f00-e3a5-4447-b4a0-a03e343af553 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:52.695837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f8f00-e3a5-4447-b4a0-a03e343af553 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:52.697126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f8f00-e3a5-4447-b4a0-a03e343af553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:52.698247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b5109d35-341c-4dca-a506-daaee6f57e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:52.700297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f8f00-e3a5-4447-b4a0-a03e343af553 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:52.717035 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1036/2072] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:52 2022] GET /identity/v3/roles => generated 1427 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:52.721487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f8f00-e3a5-4447-b4a0-a03e343af553 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:52.732854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f8f00-e3a5-4447-b4a0-a03e343af553 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:52.739677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93269b-ae9d-4ff7-b751-d5c3a5fa753a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:52.741374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93269b-ae9d-4ff7-b751-d5c3a5fa753a admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:52.741746 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93269b-ae9d-4ff7-b751-d5c3a5fa753a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:52.742096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93269b-ae9d-4ff7-b751-d5c3a5fa753a admin admin] PATH_INFO: `/v3/projects/543a07e8feba449787760847504ab114/users/4e5829db0c93486793f2b8d0482ec203/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:52.761660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93269b-ae9d-4ff7-b751-d5c3a5fa753a admin admin] RBAC: Authorizing `identity:create_grant(project_id=543a07e8feba449787760847504ab114, user_id=4e5829db0c93486793f2b8d0482ec203, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:52.765053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93269b-ae9d-4ff7-b751-d5c3a5fa753a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:52.834928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1037/2073] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 18:00:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:52.883996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1037/2074] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:52 2022] PUT /identity/v3/projects/543a07e8feba449787760847504ab114/users/4e5829db0c93486793f2b8d0482ec203/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:52.895685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5109d35-341c-4dca-a506-daaee6f57e31 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:52.913085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93269b-ae9d-4ff7-b751-d5c3a5fa753a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:52.914200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5109d35-341c-4dca-a506-daaee6f57e31 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:52.915799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-db93269b-ae9d-4ff7-b751-d5c3a5fa753a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:52.983105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1b29f-05ee-40e3-891e-2cd44cd2094a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:52.984041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1b29f-05ee-40e3-891e-2cd44cd2094a admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:52.984468 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1b29f-05ee-40e3-891e-2cd44cd2094a admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:52.984884 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1b29f-05ee-40e3-891e-2cd44cd2094a admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:52.986945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1b29f-05ee-40e3-891e-2cd44cd2094a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:52.987623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1b29f-05ee-40e3-891e-2cd44cd2094a admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:53.008001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1038/2075] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:52 2022] GET /identity/v3/roles => generated 1427 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:53.013302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1b29f-05ee-40e3-891e-2cd44cd2094a admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:53.015947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93269b-ae9d-4ff7-b751-d5c3a5fa753a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:53.022385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd5ed3-8e92-4f8c-990b-d7a419b76043 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:53.023272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd5ed3-8e92-4f8c-990b-d7a419b76043 service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:53.023972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd5ed3-8e92-4f8c-990b-d7a419b76043 service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:53.023972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd5ed3-8e92-4f8c-990b-d7a419b76043 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:53.028163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1b29f-05ee-40e3-891e-2cd44cd2094a admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:53.033897 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18b6dd-afad-4c39-92e7-d98f8d8d8415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:53.035833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18b6dd-afad-4c39-92e7-d98f8d8d8415 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:53.036107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd5ed3-8e92-4f8c-990b-d7a419b76043 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:53.036209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18b6dd-afad-4c39-92e7-d98f8d8d8415 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:53.036339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18b6dd-afad-4c39-92e7-d98f8d8d8415 admin admin] PATH_INFO: `/v3/projects/543a07e8feba449787760847504ab114/users/4e5829db0c93486793f2b8d0482ec203/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:53.037922 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd5ed3-8e92-4f8c-990b-d7a419b76043 service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:53.038934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18b6dd-afad-4c39-92e7-d98f8d8d8415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=543a07e8feba449787760847504ab114, user_id=4e5829db0c93486793f2b8d0482ec203, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:53.041706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18b6dd-afad-4c39-92e7-d98f8d8d8415 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:53.051035 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1038/2076] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:00:53.057300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1039/2077] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:53 2022] PUT /identity/v3/projects/543a07e8feba449787760847504ab114/users/4e5829db0c93486793f2b8d0482ec203/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:53.072867 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4faf2e-9662-49a1-b2fa-2779091aa1a6 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:53.073341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4faf2e-9662-49a1-b2fa-2779091aa1a6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:53.073821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4faf2e-9662-49a1-b2fa-2779091aa1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:53.116546 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1e4faf2e-9662-49a1-b2fa-2779091aa1a6 None None] MFA Rules not processed for user `4e5829db0c93486793f2b8d0482ec203`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:53.117892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1e4faf2e-9662-49a1-b2fa-2779091aa1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:53.203875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1039/2078] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:53.215591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18b6dd-afad-4c39-92e7-d98f8d8d8415 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:53.217373 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-8c18b6dd-afad-4c39-92e7-d98f8d8d8415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:53.277572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18b6dd-afad-4c39-92e7-d98f8d8d8415 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:53.281571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c6a4f-9f87-45e9-a028-cdc0ab40c362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:53.282230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c6a4f-9f87-45e9-a028-cdc0ab40c362 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:53.282555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c6a4f-9f87-45e9-a028-cdc0ab40c362 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:53.282845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c6a4f-9f87-45e9-a028-cdc0ab40c362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:53.297604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c6a4f-9f87-45e9-a028-cdc0ab40c362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:53.297604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c6a4f-9f87-45e9-a028-cdc0ab40c362 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:53.306506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1040/2079] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:53.528131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:53.541647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4faf2e-9662-49a1-b2fa-2779091aa1a6 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:53.554262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4faf2e-9662-49a1-b2fa-2779091aa1a6 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:53.558308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a56d1-6729-4edb-acf3-0557ec9796e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:53.559108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a56d1-6729-4edb-acf3-0557ec9796e8 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:53.560119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a56d1-6729-4edb-acf3-0557ec9796e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:53.560119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a56d1-6729-4edb-acf3-0557ec9796e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:53.630881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a56d1-6729-4edb-acf3-0557ec9796e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:53.632339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a56d1-6729-4edb-acf3-0557ec9796e8 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:53.644259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1040/2080] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:53.752658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c6a4f-9f87-45e9-a028-cdc0ab40c362 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:53.754590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-020c6a4f-9f87-45e9-a028-cdc0ab40c362 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:53.862993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c6a4f-9f87-45e9-a028-cdc0ab40c362 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:53.869106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:53.870075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:53.870379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:53.870658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] PATH_INFO: `/v3/users/4408a4b02d2a4f56ae17f1ec5c23bd8a` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:53.872216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4408a4b02d2a4f56ae17f1ec5c23bd8a)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:53.874992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:53.921284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f630ef1-8694-4cb4-bba1-1d3fd777f81f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:53.921754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f630ef1-8694-4cb4-bba1-1d3fd777f81f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:53.921865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f630ef1-8694-4cb4-bba1-1d3fd777f81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:53.940465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5f630ef1-8694-4cb4-bba1-1d3fd777f81f None None] MFA Rules not processed for user `556aab8b6cb04d06bbf49250a456aaf0`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:53.942303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5f630ef1-8694-4cb4-bba1-1d3fd777f81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:54.026087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1041/2081] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:54.134873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4408a4b02d2a4f56ae17f1ec5c23bd8a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:54.142082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4408a4b02d2a4f56ae17f1ec5c23bd8a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:54.142769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4408a4b02d2a4f56ae17f1ec5c23bd8a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:54.155722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4408a4b02d2a4f56ae17f1ec5c23bd8a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:54.284554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4408a4b02d2a4f56ae17f1ec5c23bd8a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:54.291732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1041/2082] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:53 2022] DELETE /identity/v3/users/4408a4b02d2a4f56ae17f1ec5c23bd8a => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:54.337874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:54.600427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f630ef1-8694-4cb4-bba1-1d3fd777f81f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:54.616407 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f630ef1-8694-4cb4-bba1-1d3fd777f81f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:54.706473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:54.708342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:54.708798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:54.709081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] PATH_INFO: `/v3/projects/5629e91c32b04a4a9bd193a0c8728274` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:54.710710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5629e91c32b04a4a9bd193a0c8728274)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:54.712683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:54.874625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5629e91c32b04a4a9bd193a0c8728274', 'request_id': 'req-af653376-08ee-4a29-a7b5-984807aee8e5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:54.982063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5629e91c32b04a4a9bd193a0c8728274', 'request_id': 'req-af653376-08ee-4a29-a7b5-984807aee8e5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:54.990412 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5629e91c32b04a4a9bd193a0c8728274', 'request_id': 'req-af653376-08ee-4a29-a7b5-984807aee8e5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:55.001879 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] The token cache is being invalidate because project 5629e91c32b04a4a9bd193a0c8728274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:55.002301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:55.003221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:55.005338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1042/2083] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:54 2022] DELETE /identity/v3/projects/5629e91c32b04a4a9bd193a0c8728274 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:55.009860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:55.014008 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:55.039183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67c1c8-d427-4fa3-a826-aaaf0c379fb0 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:55.056203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:INSERT=5,keystone:SELECT=81,keystone:DELETE=14 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:55.099553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcad9e-58f0-4bb8-96d0-b75e357eeb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:55.100444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcad9e-58f0-4bb8-96d0-b75e357eeb61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:55.100774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcad9e-58f0-4bb8-96d0-b75e357eeb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:55.101004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcad9e-58f0-4bb8-96d0-b75e357eeb61 admin admin] PATH_INFO: `/v3/users/ecad0ef54fa34683a00b8dcca96e4f08` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:55.102253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcad9e-58f0-4bb8-96d0-b75e357eeb61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecad0ef54fa34683a00b8dcca96e4f08)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:55.104251 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcad9e-58f0-4bb8-96d0-b75e357eeb61 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:55.343082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-65dcad9e-58f0-4bb8-96d0-b75e357eeb61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecad0ef54fa34683a00b8dcca96e4f08'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:55.351311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-65dcad9e-58f0-4bb8-96d0-b75e357eeb61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecad0ef54fa34683a00b8dcca96e4f08'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:55.351924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-65dcad9e-58f0-4bb8-96d0-b75e357eeb61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecad0ef54fa34683a00b8dcca96e4f08'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:55.364928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-65dcad9e-58f0-4bb8-96d0-b75e357eeb61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecad0ef54fa34683a00b8dcca96e4f08'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:55.438739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-65dcad9e-58f0-4bb8-96d0-b75e357eeb61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecad0ef54fa34683a00b8dcca96e4f08'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:55.446183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1042/2084] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:55 2022] DELETE /identity/v3/users/ecad0ef54fa34683a00b8dcca96e4f08 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:55.764387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:55.765871 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:55.779710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af653376-08ee-4a29-a7b5-984807aee8e5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:55.868610 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:55.869709 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:55.869982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:55.870254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] PATH_INFO: `/v3/projects/98b4d4734e1f47889e09ec2e8e2d96d2` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:55.872069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b4d4734e1f47889e09ec2e8e2d96d2)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:55.873852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:56.017944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b4d4734e1f47889e09ec2e8e2d96d2', 'request_id': 'req-95bfbf2a-e417-4384-85bc-10b730076e2f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:56.111497 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b4d4734e1f47889e09ec2e8e2d96d2', 'request_id': 'req-95bfbf2a-e417-4384-85bc-10b730076e2f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:56.123932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b4d4734e1f47889e09ec2e8e2d96d2', 'request_id': 'req-95bfbf2a-e417-4384-85bc-10b730076e2f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:56.136494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] The token cache is being invalidate because project 98b4d4734e1f47889e09ec2e8e2d96d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:56.136860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:56.137554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:56.143871 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1043/2085] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:55 2022] DELETE /identity/v3/projects/98b4d4734e1f47889e09ec2e8e2d96d2 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:56.149787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bd65e-b91c-4318-984f-5f718c270819 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:56.150285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bd65e-b91c-4318-984f-5f718c270819 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:56.150491 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bd65e-b91c-4318-984f-5f718c270819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:56.170029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5e9bd65e-b91c-4318-984f-5f718c270819 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:56.171437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5e9bd65e-b91c-4318-984f-5f718c270819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:56.247195 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1043/2086] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:56.247302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:56.247302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:56.249121 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bda22a-6a5d-4164-add4-9d50491da228 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:56.249486 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bda22a-6a5d-4164-add4-9d50491da228 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:56.249842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bda22a-6a5d-4164-add4-9d50491da228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:56.262338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfbf2a-e417-4384-85bc-10b730076e2f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:56.267175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:56.268143 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:56.268426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:56.268748 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] PATH_INFO: `/v3/users/84b4d6be32e94ccd80f97aa8c93efbcc` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:56.270128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b4d6be32e94ccd80f97aa8c93efbcc)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:56.272631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:56.274394 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f9bda22a-6a5d-4164-add4-9d50491da228 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:56.275831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f9bda22a-6a5d-4164-add4-9d50491da228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:56.284776 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1044/2087] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:56.288086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:56.288985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bda22a-6a5d-4164-add4-9d50491da228 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:56.298337 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bda22a-6a5d-4164-add4-9d50491da228 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:56.304243 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b11753-3d6f-4807-b3b9-71fda8af2e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:56.305152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b11753-3d6f-4807-b3b9-71fda8af2e52 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:56.305425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b11753-3d6f-4807-b3b9-71fda8af2e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:56.305760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b11753-3d6f-4807-b3b9-71fda8af2e52 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:56.306522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b11753-3d6f-4807-b3b9-71fda8af2e52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:56.306870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b11753-3d6f-4807-b3b9-71fda8af2e52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:56.308529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b11753-3d6f-4807-b3b9-71fda8af2e52 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:56.321267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1045/2088] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:56.326146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b11753-3d6f-4807-b3b9-71fda8af2e52 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:56.337637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b11753-3d6f-4807-b3b9-71fda8af2e52 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:56.342528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d9d97-02fb-4615-8cd8-e9cdbec10ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:56.343247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d9d97-02fb-4615-8cd8-e9cdbec10ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:56.343518 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d9d97-02fb-4615-8cd8-e9cdbec10ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:56.343789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d9d97-02fb-4615-8cd8-e9cdbec10ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:56.344485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d9d97-02fb-4615-8cd8-e9cdbec10ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:56.346388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d9d97-02fb-4615-8cd8-e9cdbec10ae5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:56.452397 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1046/2089] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:56 2022] POST /identity/v3/projects => generated 386 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:56.456614 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d9d97-02fb-4615-8cd8-e9cdbec10ae5 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:56.525387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d9d97-02fb-4615-8cd8-e9cdbec10ae5 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:56.531234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987764e-df92-4ef1-937d-eca4816de6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:56.531923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987764e-df92-4ef1-937d-eca4816de6ec admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:56.532203 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987764e-df92-4ef1-937d-eca4816de6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:56.532474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987764e-df92-4ef1-937d-eca4816de6ec admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:56.533173 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987764e-df92-4ef1-937d-eca4816de6ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:56.535134 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987764e-df92-4ef1-937d-eca4816de6ec admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:56.572934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b4d6be32e94ccd80f97aa8c93efbcc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:56.580118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b4d6be32e94ccd80f97aa8c93efbcc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:56.580812 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b4d6be32e94ccd80f97aa8c93efbcc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:56.593523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b4d6be32e94ccd80f97aa8c93efbcc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:56.678473 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1047/2090] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:56 2022] POST /identity/v3/users => generated 294 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:56.682907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987764e-df92-4ef1-937d-eca4816de6ec admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:56.693910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987764e-df92-4ef1-937d-eca4816de6ec admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:56.698504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdcd32-f3f4-4eab-b982-0270149aae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:56.699203 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdcd32-f3f4-4eab-b982-0270149aae47 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:56.699484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdcd32-f3f4-4eab-b982-0270149aae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:56.700230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdcd32-f3f4-4eab-b982-0270149aae47 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:56.700633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdcd32-f3f4-4eab-b982-0270149aae47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:56.702203 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdcd32-f3f4-4eab-b982-0270149aae47 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:56.718062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1048/2091] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:56 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:56.719836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdcd32-f3f4-4eab-b982-0270149aae47 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:56.730281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdcd32-f3f4-4eab-b982-0270149aae47 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:56.733018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b4d6be32e94ccd80f97aa8c93efbcc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:56.738093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c0a2b-5e52-4c7e-aad4-fd846e65e03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:56.739580 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c0a2b-5e52-4c7e-aad4-fd846e65e03d admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:56.739698 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c0a2b-5e52-4c7e-aad4-fd846e65e03d admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:56.739917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c0a2b-5e52-4c7e-aad4-fd846e65e03d admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:56.740768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c0a2b-5e52-4c7e-aad4-fd846e65e03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:56.742146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1044/2092] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:00:56 2022] DELETE /identity/v3/users/84b4d6be32e94ccd80f97aa8c93efbcc => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:56.742398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c0a2b-5e52-4c7e-aad4-fd846e65e03d admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:56.755081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1049/2093] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:56 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:56.758652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:56.760239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:56.840678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362e19e-930a-4928-9c67-d5a2a1a1188b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:56.847064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27627f-afda-4c65-96ec-432ac2b1f5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:56.848110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27627f-afda-4c65-96ec-432ac2b1f5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:56.848425 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27627f-afda-4c65-96ec-432ac2b1f5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:56.856481 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27627f-afda-4c65-96ec-432ac2b1f5e0 admin admin] PATH_INFO: `/v3/projects/a56cceea37e24839bfd403cf334f5f4d/users/68caddea0e4346df952012cd307c1f60/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:56.865649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27627f-afda-4c65-96ec-432ac2b1f5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56cceea37e24839bfd403cf334f5f4d, user_id=68caddea0e4346df952012cd307c1f60, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:56.868376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27627f-afda-4c65-96ec-432ac2b1f5e0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:57.016093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1045/2094] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:56 2022] PUT /identity/v3/projects/a56cceea37e24839bfd403cf334f5f4d/users/68caddea0e4346df952012cd307c1f60/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:57.027423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4cd6-a14b-40cc-933f-a94ad00549af None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.027712 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4cd6-a14b-40cc-933f-a94ad00549af None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.027996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4cd6-a14b-40cc-933f-a94ad00549af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.063945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27627f-afda-4c65-96ec-432ac2b1f5e0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:57.069691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0b27627f-afda-4c65-96ec-432ac2b1f5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:57.083650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-990c4cd6-a14b-40cc-933f-a94ad00549af None None] MFA Rules not processed for user `68caddea0e4346df952012cd307c1f60`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:57.091309 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-990c4cd6-a14b-40cc-933f-a94ad00549af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:57.203311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27627f-afda-4c65-96ec-432ac2b1f5e0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:57.205599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1050/2095] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:57.208781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:57.213831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.213831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.213831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa admin admin] PATH_INFO: `/v3/projects/d338e1d056d94062a703aa03ecb7edbb` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.214202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c4cd6-a14b-40cc-933f-a94ad00549af None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:57.214423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d338e1d056d94062a703aa03ecb7edbb)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:57.215206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:57.230925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c4cd6-a14b-40cc-933f-a94ad00549af None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:57.246058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea878fd-933b-4a38-8ef9-ee56433f889f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:57.246737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea878fd-933b-4a38-8ef9-ee56433f889f admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.247198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea878fd-933b-4a38-8ef9-ee56433f889f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.247700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea878fd-933b-4a38-8ef9-ee56433f889f admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.248606 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea878fd-933b-4a38-8ef9-ee56433f889f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:57.250984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea878fd-933b-4a38-8ef9-ee56433f889f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:57.347050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd338e1d056d94062a703aa03ecb7edbb', 'request_id': 'req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:57.461928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1051/2096] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:57 2022] POST /identity/v3/projects => generated 386 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:57.461928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea878fd-933b-4a38-8ef9-ee56433f889f admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:57.524088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea878fd-933b-4a38-8ef9-ee56433f889f admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:57.524617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd338e1d056d94062a703aa03ecb7edbb', 'request_id': 'req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:57.528907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca1c16-1149-4ff1-9438-3584098fff2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:57.529938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca1c16-1149-4ff1-9438-3584098fff2a admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.534302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca1c16-1149-4ff1-9438-3584098fff2a admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.534302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca1c16-1149-4ff1-9438-3584098fff2a admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.534302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca1c16-1149-4ff1-9438-3584098fff2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:57.534565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd338e1d056d94062a703aa03ecb7edbb', 'request_id': 'req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:57.534656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca1c16-1149-4ff1-9438-3584098fff2a admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:57.545143 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa admin admin] The token cache is being invalidate because project d338e1d056d94062a703aa03ecb7edbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:57.545429 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:57.546097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d9e2102d-f0ff-47b0-b9d1-5f67d8a075fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:57.547716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1046/2097] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:00:57 2022] DELETE /identity/v3/projects/d338e1d056d94062a703aa03ecb7edbb => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:57.552722 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45022c-57ef-4904-a877-7bb2489fd0a1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.553003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45022c-57ef-4904-a877-7bb2489fd0a1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.553365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45022c-57ef-4904-a877-7bb2489fd0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.571507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-0f45022c-57ef-4904-a877-7bb2489fd0a1 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:57.572697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0f45022c-57ef-4904-a877-7bb2489fd0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:57.581276 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1047/2098] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:57.590218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b8b13-7443-46b2-ad43-c12ec2a2f864 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.590500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b8b13-7443-46b2-ad43-c12ec2a2f864 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.590778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b8b13-7443-46b2-ad43-c12ec2a2f864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.611423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-319b8b13-7443-46b2-ad43-c12ec2a2f864 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:57.612677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-319b8b13-7443-46b2-ad43-c12ec2a2f864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:57.621969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1048/2099] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:57.626367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b8b13-7443-46b2-ad43-c12ec2a2f864 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:57.639507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b8b13-7443-46b2-ad43-c12ec2a2f864 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:57.642901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1052/2100] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:57 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:57.643922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca1c16-1149-4ff1-9438-3584098fff2a admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:57.646833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-acca1c16-1149-4ff1-9438-3584098fff2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:57.647769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b681647-0faa-42a1-a503-dbeee2340346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:57.648768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b681647-0faa-42a1-a503-dbeee2340346 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.649230 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b681647-0faa-42a1-a503-dbeee2340346 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.649516 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b681647-0faa-42a1-a503-dbeee2340346 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.650304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b681647-0faa-42a1-a503-dbeee2340346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:57.650616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b681647-0faa-42a1-a503-dbeee2340346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:57.652271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b681647-0faa-42a1-a503-dbeee2340346 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:57.661373 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca1c16-1149-4ff1-9438-3584098fff2a admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:57.669418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1049/2101] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:00:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:00:57.669612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bb33c-9e66-4a9f-b78d-165b88d2184b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:57.669612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bb33c-9e66-4a9f-b78d-165b88d2184b admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.669612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bb33c-9e66-4a9f-b78d-165b88d2184b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.669862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bb33c-9e66-4a9f-b78d-165b88d2184b admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.670456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b681647-0faa-42a1-a503-dbeee2340346 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:57.670732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bb33c-9e66-4a9f-b78d-165b88d2184b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:57.672459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bb33c-9e66-4a9f-b78d-165b88d2184b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:57.683198 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b681647-0faa-42a1-a503-dbeee2340346 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:57.686252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1053/2102] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:57 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:57.688226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b496b6e-8b27-423a-ac76-d8afde852580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:57.690244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b496b6e-8b27-423a-ac76-d8afde852580 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.690244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b496b6e-8b27-423a-ac76-d8afde852580 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.690244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b496b6e-8b27-423a-ac76-d8afde852580 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.690458 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b496b6e-8b27-423a-ac76-d8afde852580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:57.692594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b496b6e-8b27-423a-ac76-d8afde852580 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:57.693600 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bb33c-9e66-4a9f-b78d-165b88d2184b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:57.704665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bb33c-9e66-4a9f-b78d-165b88d2184b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:57.710593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3004ed-03dc-4dcd-8f29-9b271f64b767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:57.711573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3004ed-03dc-4dcd-8f29-9b271f64b767 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.711866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3004ed-03dc-4dcd-8f29-9b271f64b767 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.712139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3004ed-03dc-4dcd-8f29-9b271f64b767 admin admin] PATH_INFO: `/v3/projects/199bfffb12ca4b1b9b221a003f3df804/users/806a2c53ea7949e7ae94a8a56b785265/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.729244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3004ed-03dc-4dcd-8f29-9b271f64b767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=199bfffb12ca4b1b9b221a003f3df804, user_id=806a2c53ea7949e7ae94a8a56b785265, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:57.732055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3004ed-03dc-4dcd-8f29-9b271f64b767 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:57.778919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:57.787826 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1050/2103] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:00:57 2022] POST /identity/v3/projects => generated 368 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:57.791885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b496b6e-8b27-423a-ac76-d8afde852580 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:57.833445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1054/2104] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:57 2022] PUT /identity/v3/projects/199bfffb12ca4b1b9b221a003f3df804/users/806a2c53ea7949e7ae94a8a56b785265/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:00:57.837840 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3004ed-03dc-4dcd-8f29-9b271f64b767 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:57.880364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b496b6e-8b27-423a-ac76-d8afde852580 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:57.945274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3004ed-03dc-4dcd-8f29-9b271f64b767 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:57.952525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbf141-be98-4d82-88d8-24ce8837c1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:57.953587 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbf141-be98-4d82-88d8-24ce8837c1ac admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.953919 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbf141-be98-4d82-88d8-24ce8837c1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.954268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbf141-be98-4d82-88d8-24ce8837c1ac admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.955208 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbf141-be98-4d82-88d8-24ce8837c1ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:57.957101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbf141-be98-4d82-88d8-24ce8837c1ac admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:57.970420 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d5b41-760d-40de-ac9a-72495555faf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:57.970663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1055/2105] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:57 2022] GET /identity/v3/roles => generated 1427 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:57.974263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d5b41-760d-40de-ac9a-72495555faf0 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.974263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d5b41-760d-40de-ac9a-72495555faf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.974263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d5b41-760d-40de-ac9a-72495555faf0 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.974263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d5b41-760d-40de-ac9a-72495555faf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:57.975253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d5b41-760d-40de-ac9a-72495555faf0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:57.977530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbf141-be98-4d82-88d8-24ce8837c1ac admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:57.989646 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbf141-be98-4d82-88d8-24ce8837c1ac admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:57.994660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736b31e-38ba-4f78-8fcd-a47057e10de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:57.995593 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736b31e-38ba-4f78-8fcd-a47057e10de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:57.995897 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736b31e-38ba-4f78-8fcd-a47057e10de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:57.996162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736b31e-38ba-4f78-8fcd-a47057e10de0 admin admin] PATH_INFO: `/v3/projects/199bfffb12ca4b1b9b221a003f3df804/users/806a2c53ea7949e7ae94a8a56b785265/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:57.998116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736b31e-38ba-4f78-8fcd-a47057e10de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=199bfffb12ca4b1b9b221a003f3df804, user_id=806a2c53ea7949e7ae94a8a56b785265, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:58.000775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736b31e-38ba-4f78-8fcd-a47057e10de0 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:58.015656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1056/2106] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:57 2022] PUT /identity/v3/projects/199bfffb12ca4b1b9b221a003f3df804/users/806a2c53ea7949e7ae94a8a56b785265/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:58.026755 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bcc90-dd95-4f4f-82c2-a23aefeda0ae None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:58.027182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bcc90-dd95-4f4f-82c2-a23aefeda0ae None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:58.027466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bcc90-dd95-4f4f-82c2-a23aefeda0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:58.062512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c15bcc90-dd95-4f4f-82c2-a23aefeda0ae None None] MFA Rules not processed for user `806a2c53ea7949e7ae94a8a56b785265`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:58.063785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c15bcc90-dd95-4f4f-82c2-a23aefeda0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:58.131631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1051/2107] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:00:57 2022] POST /identity/v3/users => generated 285 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:00:58.135773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d5b41-760d-40de-ac9a-72495555faf0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:58.143101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1057/2108] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:58.146905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d5b41-760d-40de-ac9a-72495555faf0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:58.160378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bcc90-dd95-4f4f-82c2-a23aefeda0ae None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:58.163380 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c15bcc90-dd95-4f4f-82c2-a23aefeda0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:58.236966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6d331-5d60-41b6-813f-a3e7a8dd188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:58.237955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6d331-5d60-41b6-813f-a3e7a8dd188e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:58.238487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6d331-5d60-41b6-813f-a3e7a8dd188e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:58.239885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6d331-5d60-41b6-813f-a3e7a8dd188e admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:58.241126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6d331-5d60-41b6-813f-a3e7a8dd188e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:58.256436 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bcc90-dd95-4f4f-82c2-a23aefeda0ae None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:58.262903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abd437-189e-4c83-9e9f-42461ddd560b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:58.263782 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abd437-189e-4c83-9e9f-42461ddd560b service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:58.264395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abd437-189e-4c83-9e9f-42461ddd560b service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:58.264885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abd437-189e-4c83-9e9f-42461ddd560b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:58.265384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6d331-5d60-41b6-813f-a3e7a8dd188e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:58.267349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-85abd437-189e-4c83-9e9f-42461ddd560b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:58.291048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1052/2109] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:58 2022] GET /identity/v3/roles => generated 1427 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:58.298350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6d331-5d60-41b6-813f-a3e7a8dd188e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:58.317952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6d331-5d60-41b6-813f-a3e7a8dd188e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:58.330605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b464efc-f49c-4c04-a310-82e8e16eacdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:58.331552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b464efc-f49c-4c04-a310-82e8e16eacdd admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:58.332293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b464efc-f49c-4c04-a310-82e8e16eacdd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:58.332725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b464efc-f49c-4c04-a310-82e8e16eacdd admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:58.335915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b464efc-f49c-4c04-a310-82e8e16eacdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:58.337972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b464efc-f49c-4c04-a310-82e8e16eacdd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:58.356708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1053/2110] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:00:58 2022] GET /identity/v3/roles => generated 1427 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:00:58.360769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b464efc-f49c-4c04-a310-82e8e16eacdd admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:58.367319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abd437-189e-4c83-9e9f-42461ddd560b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:58.370144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abd437-189e-4c83-9e9f-42461ddd560b service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:58.378466 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b464efc-f49c-4c04-a310-82e8e16eacdd admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:58.388407 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df7152-fbcd-402c-832f-b2dd20927b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:58.389213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1058/2111] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:58.389590 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df7152-fbcd-402c-832f-b2dd20927b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:58.389975 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df7152-fbcd-402c-832f-b2dd20927b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:58.390389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df7152-fbcd-402c-832f-b2dd20927b87 admin admin] PATH_INFO: `/v3/projects/53f05a3845da4691a18d891d7b1bbd96/users/bda4ad5819654f6f80f84a6edf4f699a/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:58.412360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df7152-fbcd-402c-832f-b2dd20927b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f05a3845da4691a18d891d7b1bbd96, user_id=bda4ad5819654f6f80f84a6edf4f699a, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:58.415246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df7152-fbcd-402c-832f-b2dd20927b87 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:58.525783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1054/2112] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:00:58 2022] PUT /identity/v3/projects/53f05a3845da4691a18d891d7b1bbd96/users/bda4ad5819654f6f80f84a6edf4f699a/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:58.536733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f251d8e-1544-4327-bfa0-4f5ce8147674 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:58.537242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f251d8e-1544-4327-bfa0-4f5ce8147674 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:58.537617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f251d8e-1544-4327-bfa0-4f5ce8147674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:58.581242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1f251d8e-1544-4327-bfa0-4f5ce8147674 None None] MFA Rules not processed for user `bda4ad5819654f6f80f84a6edf4f699a`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:58.583520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1f251d8e-1544-4327-bfa0-4f5ce8147674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:58.667000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1059/2113] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:00:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:58.769325 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df7152-fbcd-402c-832f-b2dd20927b87 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:58.810644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:58.833612 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df7152-fbcd-402c-832f-b2dd20927b87 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:58.837820 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7315c-9b03-40ee-885c-ceab5e0c88d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:58.838567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7315c-9b03-40ee-885c-ceab5e0c88d4 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:58.838956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7315c-9b03-40ee-885c-ceab5e0c88d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:58.839305 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7315c-9b03-40ee-885c-ceab5e0c88d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:58.855126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7315c-9b03-40ee-885c-ceab5e0c88d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:58.857137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7315c-9b03-40ee-885c-ceab5e0c88d4 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:58.867180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1055/2114] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:00:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:00.412497 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:00.937133 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:01.408248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:02.053803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:02.168233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:02.976554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f251d8e-1544-4327-bfa0-4f5ce8147674 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:02.978376 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1f251d8e-1544-4327-bfa0-4f5ce8147674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:03.044851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f251d8e-1544-4327-bfa0-4f5ce8147674 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:03.049349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:03.050216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:03.050493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:03.050762 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] PATH_INFO: `/v3/users/d0bdd63e8e0f466f8f38ce6fd0bb2d29` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:03.052565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0bdd63e8e0f466f8f38ce6fd0bb2d29)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:03.054572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:03.118894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:03.285039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0bdd63e8e0f466f8f38ce6fd0bb2d29'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:03.294317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0bdd63e8e0f466f8f38ce6fd0bb2d29'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:03.295422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0bdd63e8e0f466f8f38ce6fd0bb2d29'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:03.319870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0bdd63e8e0f466f8f38ce6fd0bb2d29'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:03.490637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0bdd63e8e0f466f8f38ce6fd0bb2d29'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:03.497166 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1060/2115] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:02 2022] DELETE /identity/v3/users/d0bdd63e8e0f466f8f38ce6fd0bb2d29 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:03.501984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9ad0f-bd07-445c-8ac3-975e5e7462b1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:03.502268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9ad0f-bd07-445c-8ac3-975e5e7462b1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:03.502543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9ad0f-bd07-445c-8ac3-975e5e7462b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:03.522736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-56e9ad0f-bd07-445c-8ac3-975e5e7462b1 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:03.523955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-56e9ad0f-bd07-445c-8ac3-975e5e7462b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:03.586753 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1056/2116] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:03.596635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:03.598528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:03.661486 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce913c3-40b9-430c-9e3a-f939cb8b0bff admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:03.665411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2732c5a-e42f-4feb-95ac-085e9a88a62b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:03.666346 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2732c5a-e42f-4feb-95ac-085e9a88a62b service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:03.666618 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2732c5a-e42f-4feb-95ac-085e9a88a62b service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:03.666920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2732c5a-e42f-4feb-95ac-085e9a88a62b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:03.676556 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2732c5a-e42f-4feb-95ac-085e9a88a62b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:03.678076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2732c5a-e42f-4feb-95ac-085e9a88a62b service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:03.687707 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1061/2117] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:03.751557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:04.167290 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9ad0f-bd07-445c-8ac3-975e5e7462b1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:04.168866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-56e9ad0f-bd07-445c-8ac3-975e5e7462b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:04.181257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9ad0f-bd07-445c-8ac3-975e5e7462b1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:04.186259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:04.187171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:04.187448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:04.187720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] PATH_INFO: `/v3/projects/f5ad3445821c4925900089667a19f58c` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:04.188924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ad3445821c4925900089667a19f58c)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:04.190843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:04.340394 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ad3445821c4925900089667a19f58c', 'request_id': 'req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:04.455154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ad3445821c4925900089667a19f58c', 'request_id': 'req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:04.458961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:04.465600 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ad3445821c4925900089667a19f58c', 'request_id': 'req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:04.484932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] The token cache is being invalidate because project f5ad3445821c4925900089667a19f58c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:04.485349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:04.486085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:04.488215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1057/2118] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:04 2022] DELETE /identity/v3/projects/f5ad3445821c4925900089667a19f58c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:04.492638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2732c5a-e42f-4feb-95ac-085e9a88a62b service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:04.511433 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2732c5a-e42f-4feb-95ac-085e9a88a62b service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:04.596398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89aba5-e7a7-4c20-a1ba-22b35634ea43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:04.597287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89aba5-e7a7-4c20-a1ba-22b35634ea43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:04.597569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89aba5-e7a7-4c20-a1ba-22b35634ea43 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:04.598229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89aba5-e7a7-4c20-a1ba-22b35634ea43 admin admin] PATH_INFO: `/v3/users/4e5829db0c93486793f2b8d0482ec203` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:04.599591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89aba5-e7a7-4c20-a1ba-22b35634ea43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e5829db0c93486793f2b8d0482ec203)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:04.602483 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89aba5-e7a7-4c20-a1ba-22b35634ea43 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:04.831917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fd89aba5-e7a7-4c20-a1ba-22b35634ea43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e5829db0c93486793f2b8d0482ec203'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:04.841237 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fd89aba5-e7a7-4c20-a1ba-22b35634ea43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e5829db0c93486793f2b8d0482ec203'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:04.841958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fd89aba5-e7a7-4c20-a1ba-22b35634ea43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e5829db0c93486793f2b8d0482ec203'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:04.855552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fd89aba5-e7a7-4c20-a1ba-22b35634ea43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e5829db0c93486793f2b8d0482ec203'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:04.945048 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fd89aba5-e7a7-4c20-a1ba-22b35634ea43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e5829db0c93486793f2b8d0482ec203'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:04.957255 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1062/2119] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:04 2022] DELETE /identity/v3/users/4e5829db0c93486793f2b8d0482ec203 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:05.016926 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:05.464581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:05.466172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:05.482348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc8d6e-bca8-496a-bad8-05eb36a2ca34 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:05.549435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b54f00-3b03-4c58-ba56-f706ab0f1508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:05.550315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b54f00-3b03-4c58-ba56-f706ab0f1508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:05.550603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b54f00-3b03-4c58-ba56-f706ab0f1508 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:05.550893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b54f00-3b03-4c58-ba56-f706ab0f1508 admin admin] PATH_INFO: `/v3/projects/543a07e8feba449787760847504ab114` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:05.552631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b54f00-3b03-4c58-ba56-f706ab0f1508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=543a07e8feba449787760847504ab114)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:05.554744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b54f00-3b03-4c58-ba56-f706ab0f1508 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:05.698254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d8b54f00-3b03-4c58-ba56-f706ab0f1508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '543a07e8feba449787760847504ab114', 'request_id': 'req-d8b54f00-3b03-4c58-ba56-f706ab0f1508'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:05.785724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:05.787864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d8b54f00-3b03-4c58-ba56-f706ab0f1508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '543a07e8feba449787760847504ab114', 'request_id': 'req-d8b54f00-3b03-4c58-ba56-f706ab0f1508'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:05.795565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d8b54f00-3b03-4c58-ba56-f706ab0f1508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '543a07e8feba449787760847504ab114', 'request_id': 'req-d8b54f00-3b03-4c58-ba56-f706ab0f1508'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:05.809995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d8b54f00-3b03-4c58-ba56-f706ab0f1508 admin admin] The token cache is being invalidate because project 543a07e8feba449787760847504ab114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:05.810321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d8b54f00-3b03-4c58-ba56-f706ab0f1508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:05.811162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d8b54f00-3b03-4c58-ba56-f706ab0f1508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:05.817787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1058/2120] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:05 2022] DELETE /identity/v3/projects/543a07e8feba449787760847504ab114 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:05.819954 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f13367-0bd2-4150-9d8a-6862756302aa None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:05.820917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f13367-0bd2-4150-9d8a-6862756302aa None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:05.821399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f13367-0bd2-4150-9d8a-6862756302aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:05.851598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f1f13367-0bd2-4150-9d8a-6862756302aa None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:05.853118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f1f13367-0bd2-4150-9d8a-6862756302aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:05.929167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1063/2121] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:05.937479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3c253-8b2b-4399-a2c8-77c16ccec852 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:05.937834 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3c253-8b2b-4399-a2c8-77c16ccec852 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:05.938201 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3c253-8b2b-4399-a2c8-77c16ccec852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:05.963127 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2df3c253-8b2b-4399-a2c8-77c16ccec852 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:05.964725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2df3c253-8b2b-4399-a2c8-77c16ccec852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:05.986315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1059/2122] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:05.990918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f13367-0bd2-4150-9d8a-6862756302aa None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:06.001146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f13367-0bd2-4150-9d8a-6862756302aa None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:06.007213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79536b5-774c-47ca-845d-9629e49a4d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:06.007900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79536b5-774c-47ca-845d-9629e49a4d53 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:06.008248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79536b5-774c-47ca-845d-9629e49a4d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:06.008487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79536b5-774c-47ca-845d-9629e49a4d53 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:06.009220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79536b5-774c-47ca-845d-9629e49a4d53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:01:06.009554 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79536b5-774c-47ca-845d-9629e49a4d53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:06.011223 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79536b5-774c-47ca-845d-9629e49a4d53 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:06.024027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1064/2123] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:01:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:01:06.028468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3c253-8b2b-4399-a2c8-77c16ccec852 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:06.038638 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3c253-8b2b-4399-a2c8-77c16ccec852 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:06.043814 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8c5e9-2ccb-4b79-9900-118c22fd0791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:06.044914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8c5e9-2ccb-4b79-9900-118c22fd0791 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:06.045400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8c5e9-2ccb-4b79-9900-118c22fd0791 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:06.045895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8c5e9-2ccb-4b79-9900-118c22fd0791 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:06.048201 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8c5e9-2ccb-4b79-9900-118c22fd0791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:06.050538 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8c5e9-2ccb-4b79-9900-118c22fd0791 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:06.161473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1060/2124] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:01:06 2022] POST /identity/v3/projects => generated 386 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:06.166062 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79536b5-774c-47ca-845d-9629e49a4d53 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:06.176329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79536b5-774c-47ca-845d-9629e49a4d53 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:06.235534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7e48f-287a-48fd-9a7f-b33fc90c1ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:06.236212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7e48f-287a-48fd-9a7f-b33fc90c1ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:06.236494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7e48f-287a-48fd-9a7f-b33fc90c1ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:06.236771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7e48f-287a-48fd-9a7f-b33fc90c1ed5 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:06.237520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7e48f-287a-48fd-9a7f-b33fc90c1ed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:06.239781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7e48f-287a-48fd-9a7f-b33fc90c1ed5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:06.359789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1065/2125] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:01:06 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:06.364279 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8c5e9-2ccb-4b79-9900-118c22fd0791 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:06.374766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8c5e9-2ccb-4b79-9900-118c22fd0791 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:06.380308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c81d7-db45-4469-b31e-a03d2a8c8186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:06.380984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c81d7-db45-4469-b31e-a03d2a8c8186 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:06.381263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c81d7-db45-4469-b31e-a03d2a8c8186 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:06.381541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c81d7-db45-4469-b31e-a03d2a8c8186 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:06.389824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c81d7-db45-4469-b31e-a03d2a8c8186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:06.389824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c81d7-db45-4469-b31e-a03d2a8c8186 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:06.395008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1061/2126] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:06 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:06.400000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7e48f-287a-48fd-9a7f-b33fc90c1ed5 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:06.410669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7e48f-287a-48fd-9a7f-b33fc90c1ed5 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:06.415614 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040a7f5-2c37-47cf-b308-1ded20e068f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:06.416282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040a7f5-2c37-47cf-b308-1ded20e068f5 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:06.416564 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040a7f5-2c37-47cf-b308-1ded20e068f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:06.416835 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040a7f5-2c37-47cf-b308-1ded20e068f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:06.417582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040a7f5-2c37-47cf-b308-1ded20e068f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:06.419265 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040a7f5-2c37-47cf-b308-1ded20e068f5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:06.431242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1066/2127] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:06 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:06.435340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c81d7-db45-4469-b31e-a03d2a8c8186 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:06.446957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c81d7-db45-4469-b31e-a03d2a8c8186 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:06.451865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4f11b-877f-48b2-b3ad-c98edd5c8804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:06.452757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4f11b-877f-48b2-b3ad-c98edd5c8804 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:06.453035 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4f11b-877f-48b2-b3ad-c98edd5c8804 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:06.453305 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4f11b-877f-48b2-b3ad-c98edd5c8804 admin admin] PATH_INFO: `/v3/projects/2da8de4857cf4231bc0227e2bc1b48d3/users/050a9250c2c84ca5abe2c39ed035ca53/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:06.470055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4f11b-877f-48b2-b3ad-c98edd5c8804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da8de4857cf4231bc0227e2bc1b48d3, user_id=050a9250c2c84ca5abe2c39ed035ca53, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:06.472998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4f11b-877f-48b2-b3ad-c98edd5c8804 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:06.604048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1062/2128] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:06 2022] PUT /identity/v3/projects/2da8de4857cf4231bc0227e2bc1b48d3/users/050a9250c2c84ca5abe2c39ed035ca53/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:06.616483 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2b27b-f53d-4b0c-9c4a-ac06dcef560c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:06.616769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2b27b-f53d-4b0c-9c4a-ac06dcef560c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:06.617063 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2b27b-f53d-4b0c-9c4a-ac06dcef560c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:06.646706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-63b2b27b-f53d-4b0c-9c4a-ac06dcef560c None None] MFA Rules not processed for user `050a9250c2c84ca5abe2c39ed035ca53`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:06.647983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-63b2b27b-f53d-4b0c-9c4a-ac06dcef560c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:06.688174 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:06.714936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1067/2129] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:01:06.726017 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4f11b-877f-48b2-b3ad-c98edd5c8804 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:06.727743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2fc4f11b-877f-48b2-b3ad-c98edd5c8804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:06.786926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4f11b-877f-48b2-b3ad-c98edd5c8804 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:06.790685 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87132449-5a1b-4a59-b7a1-e37e681ec4d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:06.791399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87132449-5a1b-4a59-b7a1-e37e681ec4d8 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:06.791799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87132449-5a1b-4a59-b7a1-e37e681ec4d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:06.792164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87132449-5a1b-4a59-b7a1-e37e681ec4d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:06.802423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87132449-5a1b-4a59-b7a1-e37e681ec4d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:06.805034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87132449-5a1b-4a59-b7a1-e37e681ec4d8 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:06.815331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1063/2130] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:07.309364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2b27b-f53d-4b0c-9c4a-ac06dcef560c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:07.310866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-63b2b27b-f53d-4b0c-9c4a-ac06dcef560c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:07.322623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2b27b-f53d-4b0c-9c4a-ac06dcef560c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:07.326206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4dcd8a-8a38-4c1c-87bf-e5626c3acd38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:07.326880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dcd8a-8a38-4c1c-87bf-e5626c3acd38 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:07.327199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dcd8a-8a38-4c1c-87bf-e5626c3acd38 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:07.327422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dcd8a-8a38-4c1c-87bf-e5626c3acd38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:07.329200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ae4dcd8a-8a38-4c1c-87bf-e5626c3acd38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:07.399110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4dcd8a-8a38-4c1c-87bf-e5626c3acd38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:07.400545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4dcd8a-8a38-4c1c-87bf-e5626c3acd38 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:07.410695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1068/2131] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:07.610769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:08.783032 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:09.019978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:09.471444 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:10.803209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:11.508312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:11.859779 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:11.914284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:12.116783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:12.179329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:12.209200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:12.460689 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:12.496913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:12.662742 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:12.696779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:12.928670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:12.953823 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:13.141856 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:13.174560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:13.371154 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:13.405265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:13.576985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:13.688721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:13.782584 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:13.871224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:13.891463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:14.031243 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:14.062467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:14.191355 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:14.219641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:14.348666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:14.465871 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:14.514033 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:14.691038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:14.791669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:15.009560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:15.120272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:15.159706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:15.322842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:15.351340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:15.471154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:20.616574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87132449-5a1b-4a59-b7a1-e37e681ec4d8 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:20.618402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-87132449-5a1b-4a59-b7a1-e37e681ec4d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:20.692711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87132449-5a1b-4a59-b7a1-e37e681ec4d8 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:20.699215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:20.700242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:20.700574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:20.700855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] PATH_INFO: `/v3/users/bda4ad5819654f6f80f84a6edf4f699a` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:20.702131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda4ad5819654f6f80f84a6edf4f699a)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:20.704116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:20.902214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:20.917576 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda4ad5819654f6f80f84a6edf4f699a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:20.928339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda4ad5819654f6f80f84a6edf4f699a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:20.941741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda4ad5819654f6f80f84a6edf4f699a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:20.967361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda4ad5819654f6f80f84a6edf4f699a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:21.065907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda4ad5819654f6f80f84a6edf4f699a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:21.073457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1064/2132] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:20 2022] DELETE /identity/v3/users/bda4ad5819654f6f80f84a6edf4f699a => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:21.078492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308eccf-8c87-4f2f-89dd-e9d5f349f2bc None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:21.079004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308eccf-8c87-4f2f-89dd-e9d5f349f2bc None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:21.079133 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308eccf-8c87-4f2f-89dd-e9d5f349f2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:21.098935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b308eccf-8c87-4f2f-89dd-e9d5f349f2bc None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:21.104699 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b308eccf-8c87-4f2f-89dd-e9d5f349f2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:21.117869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:21.172559 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1069/2133] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:21.183351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:21.185626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:21.263553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449a1a4-60eb-44d8-93aa-70e01167897b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:21.267375 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf3bde-e6bc-46a4-a0b1-b3b71a6a190a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:21.268047 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf3bde-e6bc-46a4-a0b1-b3b71a6a190a service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:21.268334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf3bde-e6bc-46a4-a0b1-b3b71a6a190a service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:21.268607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf3bde-e6bc-46a4-a0b1-b3b71a6a190a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:21.280735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf3bde-e6bc-46a4-a0b1-b3b71a6a190a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:21.285479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf3bde-e6bc-46a4-a0b1-b3b71a6a190a service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:21.303460 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1065/2134] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:21.315488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308eccf-8c87-4f2f-89dd-e9d5f349f2bc None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:21.319720 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b308eccf-8c87-4f2f-89dd-e9d5f349f2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:21.330528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308eccf-8c87-4f2f-89dd-e9d5f349f2bc None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:21.335636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:21.336596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:21.336954 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:21.337882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] PATH_INFO: `/v3/users/68caddea0e4346df952012cd307c1f60` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:21.339876 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=68caddea0e4346df952012cd307c1f60)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:21.341911 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:21.602325 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68caddea0e4346df952012cd307c1f60'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:21.612769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68caddea0e4346df952012cd307c1f60'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:21.613820 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68caddea0e4346df952012cd307c1f60'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:21.630034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68caddea0e4346df952012cd307c1f60'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:21.734227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68caddea0e4346df952012cd307c1f60'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:21.744410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1070/2135] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:21 2022] DELETE /identity/v3/users/68caddea0e4346df952012cd307c1f60 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:01:21.746733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b66119-d034-4468-bbe5-6ab6abbd1ca9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:21.747262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b66119-d034-4468-bbe5-6ab6abbd1ca9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:21.747356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b66119-d034-4468-bbe5-6ab6abbd1ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:21.776830 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-42b66119-d034-4468-bbe5-6ab6abbd1ca9 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:21.778179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-42b66119-d034-4468-bbe5-6ab6abbd1ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:21.810833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:21.812960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:21.865931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1066/2136] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:01:21.878974 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b66119-d034-4468-bbe5-6ab6abbd1ca9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:21.880338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-42b66119-d034-4468-bbe5-6ab6abbd1ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:21.890098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e3b77-6430-4b0d-9b77-0f510dad0a8b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:21.895167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9e4be-68cc-4163-9fc3-c9bea0439c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:21.896089 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9e4be-68cc-4163-9fc3-c9bea0439c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:21.896356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9e4be-68cc-4163-9fc3-c9bea0439c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:21.896626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9e4be-68cc-4163-9fc3-c9bea0439c38 admin admin] PATH_INFO: `/v3/users/050a9250c2c84ca5abe2c39ed035ca53` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:21.897811 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9e4be-68cc-4163-9fc3-c9bea0439c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=050a9250c2c84ca5abe2c39ed035ca53)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:21.899892 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9e4be-68cc-4163-9fc3-c9bea0439c38 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:21.968228 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b66119-d034-4468-bbe5-6ab6abbd1ca9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:21.978957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb474d-e0ab-40cc-9f71-a3d90bbc74d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:21.978957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb474d-e0ab-40cc-9f71-a3d90bbc74d9 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:21.978957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb474d-e0ab-40cc-9f71-a3d90bbc74d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:21.978957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb474d-e0ab-40cc-9f71-a3d90bbc74d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:21.988556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:21.988556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb474d-e0ab-40cc-9f71-a3d90bbc74d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:21.988556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb474d-e0ab-40cc-9f71-a3d90bbc74d9 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:22.000337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1067/2137] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:01:22.001168 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb474d-e0ab-40cc-9f71-a3d90bbc74d9 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:22.002741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a6eb474d-e0ab-40cc-9f71-a3d90bbc74d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:22.018790 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb474d-e0ab-40cc-9f71-a3d90bbc74d9 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:22.024954 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee655f-405a-4444-83b1-5a95c0ac618a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:22.025444 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee655f-405a-4444-83b1-5a95c0ac618a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:22.025722 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee655f-405a-4444-83b1-5a95c0ac618a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:22.026039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee655f-405a-4444-83b1-5a95c0ac618a admin admin] PATH_INFO: `/v3/projects/53f05a3845da4691a18d891d7b1bbd96` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:22.028427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee655f-405a-4444-83b1-5a95c0ac618a admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f05a3845da4691a18d891d7b1bbd96)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:22.030512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee655f-405a-4444-83b1-5a95c0ac618a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:22.153278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:22.173527 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5ff9e4be-68cc-4163-9fc3-c9bea0439c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '050a9250c2c84ca5abe2c39ed035ca53'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:22.182998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f7ee655f-405a-4444-83b1-5a95c0ac618a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f05a3845da4691a18d891d7b1bbd96', 'request_id': 'req-f7ee655f-405a-4444-83b1-5a95c0ac618a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:22.184728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5ff9e4be-68cc-4163-9fc3-c9bea0439c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '050a9250c2c84ca5abe2c39ed035ca53'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:22.186092 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5ff9e4be-68cc-4163-9fc3-c9bea0439c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '050a9250c2c84ca5abe2c39ed035ca53'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:22.199397 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5ff9e4be-68cc-4163-9fc3-c9bea0439c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '050a9250c2c84ca5abe2c39ed035ca53'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:22.292120 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f7ee655f-405a-4444-83b1-5a95c0ac618a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f05a3845da4691a18d891d7b1bbd96', 'request_id': 'req-f7ee655f-405a-4444-83b1-5a95c0ac618a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:22.304971 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f7ee655f-405a-4444-83b1-5a95c0ac618a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f05a3845da4691a18d891d7b1bbd96', 'request_id': 'req-f7ee655f-405a-4444-83b1-5a95c0ac618a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:22.336161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f7ee655f-405a-4444-83b1-5a95c0ac618a admin admin] The token cache is being invalidate because project 53f05a3845da4691a18d891d7b1bbd96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:22.336735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f7ee655f-405a-4444-83b1-5a95c0ac618a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:22.338299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f7ee655f-405a-4444-83b1-5a95c0ac618a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:22.344728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1068/2138] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:22 2022] DELETE /identity/v3/projects/53f05a3845da4691a18d891d7b1bbd96 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:22.349535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5ff9e4be-68cc-4163-9fc3-c9bea0439c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '050a9250c2c84ca5abe2c39ed035ca53'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:22.351268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf6845-932e-44a0-84cb-05d08f052684 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:22.351881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf6845-932e-44a0-84cb-05d08f052684 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:22.352362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf6845-932e-44a0-84cb-05d08f052684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:22.361520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1071/2139] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:21 2022] DELETE /identity/v3/users/050a9250c2c84ca5abe2c39ed035ca53 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:22.366241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8840953-7792-4b00-b5a4-d1fe74db91f0 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:22.368236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8840953-7792-4b00-b5a4-d1fe74db91f0 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:22.368529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8840953-7792-4b00-b5a4-d1fe74db91f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:22.392339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3ecf6845-932e-44a0-84cb-05d08f052684 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:22.396126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3ecf6845-932e-44a0-84cb-05d08f052684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:22.399123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f8840953-7792-4b00-b5a4-d1fe74db91f0 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:22.400603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f8840953-7792-4b00-b5a4-d1fe74db91f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:22.524785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1069/2140] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:22.538984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a3a41-af47-4337-9d1f-99bcb36b4d1e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:22.539289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1072/2141] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:22.539668 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a3a41-af47-4337-9d1f-99bcb36b4d1e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:22.540240 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a3a41-af47-4337-9d1f-99bcb36b4d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:22.558620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8840953-7792-4b00-b5a4-d1fe74db91f0 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:22.561391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f8840953-7792-4b00-b5a4-d1fe74db91f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:22.574409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-900a3a41-af47-4337-9d1f-99bcb36b4d1e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:22.576442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-900a3a41-af47-4337-9d1f-99bcb36b4d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:22.591524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1070/2142] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:22.595768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a3a41-af47-4337-9d1f-99bcb36b4d1e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:22.606656 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a3a41-af47-4337-9d1f-99bcb36b4d1e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:22.615890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328b97d-d51d-4d6f-b25c-2f9e0fd5da27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:22.616911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328b97d-d51d-4d6f-b25c-2f9e0fd5da27 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:22.617437 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328b97d-d51d-4d6f-b25c-2f9e0fd5da27 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:22.618479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328b97d-d51d-4d6f-b25c-2f9e0fd5da27 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:22.618603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328b97d-d51d-4d6f-b25c-2f9e0fd5da27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:01:22.618905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328b97d-d51d-4d6f-b25c-2f9e0fd5da27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:22.622206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328b97d-d51d-4d6f-b25c-2f9e0fd5da27 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:22.642452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1071/2143] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:01:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:01:22.648534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328b97d-d51d-4d6f-b25c-2f9e0fd5da27 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:22.668894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328b97d-d51d-4d6f-b25c-2f9e0fd5da27 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:22.679050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303b5d5-9849-4981-b12e-9d5ab2c7d132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:22.679050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303b5d5-9849-4981-b12e-9d5ab2c7d132 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:22.679050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303b5d5-9849-4981-b12e-9d5ab2c7d132 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:22.679050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303b5d5-9849-4981-b12e-9d5ab2c7d132 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:22.679472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303b5d5-9849-4981-b12e-9d5ab2c7d132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:22.680667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303b5d5-9849-4981-b12e-9d5ab2c7d132 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:22.687668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8840953-7792-4b00-b5a4-d1fe74db91f0 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:22.706242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c794787-78df-4cf1-b54d-53fba0adfe64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:22.706965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c794787-78df-4cf1-b54d-53fba0adfe64 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:22.707252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c794787-78df-4cf1-b54d-53fba0adfe64 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:22.707528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c794787-78df-4cf1-b54d-53fba0adfe64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:22.725172 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c794787-78df-4cf1-b54d-53fba0adfe64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:22.726643 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c794787-78df-4cf1-b54d-53fba0adfe64 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:22.739983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1073/2144] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:01:22.740700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c794787-78df-4cf1-b54d-53fba0adfe64 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:22.742060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3c794787-78df-4cf1-b54d-53fba0adfe64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:22.766683 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c794787-78df-4cf1-b54d-53fba0adfe64 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:22.772058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:22.773122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:22.773548 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:22.773821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] PATH_INFO: `/v3/projects/a56cceea37e24839bfd403cf334f5f4d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:22.775350 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a56cceea37e24839bfd403cf334f5f4d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:22.778263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:22.816590 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1072/2145] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:01:22 2022] POST /identity/v3/projects => generated 376 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:22.821066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303b5d5-9849-4981-b12e-9d5ab2c7d132 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:22.875601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:22.897333 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303b5d5-9849-4981-b12e-9d5ab2c7d132 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:22.902809 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc85931-db5c-4f38-b613-45d142cbef93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:22.903537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc85931-db5c-4f38-b613-45d142cbef93 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:22.908235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc85931-db5c-4f38-b613-45d142cbef93 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:22.908235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc85931-db5c-4f38-b613-45d142cbef93 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:22.908235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc85931-db5c-4f38-b613-45d142cbef93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:22.908235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc85931-db5c-4f38-b613-45d142cbef93 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:22.982228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:22.985358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a56cceea37e24839bfd403cf334f5f4d', 'request_id': 'req-0a45cb6c-ff2b-4187-8551-d9c1447134c3'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:22.993786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1073/2146] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:01:22 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:01:22.999827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc85931-db5c-4f38-b613-45d142cbef93 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:23.013131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc85931-db5c-4f38-b613-45d142cbef93 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:23.080203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60855a7-795f-44e5-bd79-36ea95a96cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:23.081215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60855a7-795f-44e5-bd79-36ea95a96cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:23.081354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60855a7-795f-44e5-bd79-36ea95a96cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:23.081582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60855a7-795f-44e5-bd79-36ea95a96cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:23.082345 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60855a7-795f-44e5-bd79-36ea95a96cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:23.083980 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60855a7-795f-44e5-bd79-36ea95a96cd5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:23.100732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a56cceea37e24839bfd403cf334f5f4d', 'request_id': 'req-0a45cb6c-ff2b-4187-8551-d9c1447134c3'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.105621 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1074/2147] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:22 2022] GET /identity/v3/roles => generated 1427 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:01:23.109609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60855a7-795f-44e5-bd79-36ea95a96cd5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:23.112868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a56cceea37e24839bfd403cf334f5f4d', 'request_id': 'req-0a45cb6c-ff2b-4187-8551-d9c1447134c3'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.122443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60855a7-795f-44e5-bd79-36ea95a96cd5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:23.126455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] The token cache is being invalidate because project a56cceea37e24839bfd403cf334f5f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:23.126878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.128022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc58c3-8ee4-4d40-ad29-410363af05fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:23.128174 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.128790 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc58c3-8ee4-4d40-ad29-410363af05fe admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:23.129068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc58c3-8ee4-4d40-ad29-410363af05fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:23.129339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc58c3-8ee4-4d40-ad29-410363af05fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:23.130158 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1074/2148] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:22 2022] DELETE /identity/v3/projects/a56cceea37e24839bfd403cf334f5f4d => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:23.130266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc58c3-8ee4-4d40-ad29-410363af05fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:23.132386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc58c3-8ee4-4d40-ad29-410363af05fe admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:23.134029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:23.136566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:23.145356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1075/2149] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:23 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:23.149365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc58c3-8ee4-4d40-ad29-410363af05fe admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:23.151789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45cb6c-ff2b-4187-8551-d9c1447134c3 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:23.152137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-31fc58c3-8ee4-4d40-ad29-410363af05fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:23.161881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc4af5-067c-4d00-9b70-d985c94195a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:23.163470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc4af5-067c-4d00-9b70-d985c94195a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:23.163905 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc4af5-067c-4d00-9b70-d985c94195a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:23.164181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc4af5-067c-4d00-9b70-d985c94195a5 admin admin] PATH_INFO: `/v3/users/806a2c53ea7949e7ae94a8a56b785265` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:23.165543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc4af5-067c-4d00-9b70-d985c94195a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=806a2c53ea7949e7ae94a8a56b785265)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:23.167440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc58c3-8ee4-4d40-ad29-410363af05fe admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:23.168242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc4af5-067c-4d00-9b70-d985c94195a5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:23.175066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ec75d-611a-4199-8652-10430909032a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:23.179680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ec75d-611a-4199-8652-10430909032a admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:23.179969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ec75d-611a-4199-8652-10430909032a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:23.180248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ec75d-611a-4199-8652-10430909032a admin admin] PATH_INFO: `/v3/projects/588c313eba2e4469bc1206ab2794a2c2/users/841c59c497444d23819c386349fc4e3d/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:23.200332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ec75d-611a-4199-8652-10430909032a admin admin] RBAC: Authorizing `identity:create_grant(project_id=588c313eba2e4469bc1206ab2794a2c2, user_id=841c59c497444d23819c386349fc4e3d, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:23.203229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ec75d-611a-4199-8652-10430909032a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:23.346016 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1076/2150] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:23 2022] PUT /identity/v3/projects/588c313eba2e4469bc1206ab2794a2c2/users/841c59c497444d23819c386349fc4e3d/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:01:23.348330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ec75d-611a-4199-8652-10430909032a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:23.348588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-425ec75d-611a-4199-8652-10430909032a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:23.425836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ec75d-611a-4199-8652-10430909032a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:23.430800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c62939-ae51-4487-951e-2010355d8a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:23.431697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c62939-ae51-4487-951e-2010355d8a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:23.431974 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c62939-ae51-4487-951e-2010355d8a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:23.432280 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c62939-ae51-4487-951e-2010355d8a51 admin admin] PATH_INFO: `/v3/projects/2da8de4857cf4231bc0227e2bc1b48d3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:23.436546 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c62939-ae51-4487-951e-2010355d8a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da8de4857cf4231bc0227e2bc1b48d3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:23.436546 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c62939-ae51-4487-951e-2010355d8a51 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:23.520193 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-00fc4af5-067c-4d00-9b70-d985c94195a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '806a2c53ea7949e7ae94a8a56b785265'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.527880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-00fc4af5-067c-4d00-9b70-d985c94195a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '806a2c53ea7949e7ae94a8a56b785265'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.531920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-00fc4af5-067c-4d00-9b70-d985c94195a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '806a2c53ea7949e7ae94a8a56b785265'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.541822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-00fc4af5-067c-4d00-9b70-d985c94195a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '806a2c53ea7949e7ae94a8a56b785265'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.621294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-44c62939-ae51-4487-951e-2010355d8a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da8de4857cf4231bc0227e2bc1b48d3', 'request_id': 'req-44c62939-ae51-4487-951e-2010355d8a51'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.648590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-00fc4af5-067c-4d00-9b70-d985c94195a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '806a2c53ea7949e7ae94a8a56b785265'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.656574 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1075/2151] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:23 2022] DELETE /identity/v3/users/806a2c53ea7949e7ae94a8a56b785265 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:23.659674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7e39e-00ce-4175-96c7-75cafffabdc1 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:23.660051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7e39e-00ce-4175-96c7-75cafffabdc1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:23.660227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7e39e-00ce-4175-96c7-75cafffabdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:23.689778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-4ef7e39e-00ce-4175-96c7-75cafffabdc1 None None] MFA Rules not processed for user `841c59c497444d23819c386349fc4e3d`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:23.691047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4ef7e39e-00ce-4175-96c7-75cafffabdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:23.720035 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-44c62939-ae51-4487-951e-2010355d8a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da8de4857cf4231bc0227e2bc1b48d3', 'request_id': 'req-44c62939-ae51-4487-951e-2010355d8a51'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.727863 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-44c62939-ae51-4487-951e-2010355d8a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da8de4857cf4231bc0227e2bc1b48d3', 'request_id': 'req-44c62939-ae51-4487-951e-2010355d8a51'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.755284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-44c62939-ae51-4487-951e-2010355d8a51 admin admin] The token cache is being invalidate because project 2da8de4857cf4231bc0227e2bc1b48d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:23.755571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-44c62939-ae51-4487-951e-2010355d8a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.756257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-44c62939-ae51-4487-951e-2010355d8a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:23.767063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1077/2152] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:23 2022] DELETE /identity/v3/projects/2da8de4857cf4231bc0227e2bc1b48d3 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:23.767063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e119ab7-f4bf-4a58-bd0a-b21059446589 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:23.767063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e119ab7-f4bf-4a58-bd0a-b21059446589 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:23.767063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e119ab7-f4bf-4a58-bd0a-b21059446589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:23.788959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1076/2153] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:23.800749 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7e39e-00ce-4175-96c7-75cafffabdc1 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:23.804843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4ef7e39e-00ce-4175-96c7-75cafffabdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:23.819530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:23.828447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5e119ab7-f4bf-4a58-bd0a-b21059446589 None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:23.830299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5e119ab7-f4bf-4a58-bd0a-b21059446589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:23.883313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:23.975337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1078/2154] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 18:01:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:23.975337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e119ab7-f4bf-4a58-bd0a-b21059446589 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:23.980826 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5e119ab7-f4bf-4a58-bd0a-b21059446589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:23.985446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7e39e-00ce-4175-96c7-75cafffabdc1 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:24.003985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcee7b-fe20-4202-87ad-60bc4d4a3c0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:24.005326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcee7b-fe20-4202-87ad-60bc4d4a3c0d service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:24.005715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcee7b-fe20-4202-87ad-60bc4d4a3c0d service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:24.006082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcee7b-fe20-4202-87ad-60bc4d4a3c0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:24.017946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fcee7b-fe20-4202-87ad-60bc4d4a3c0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:24.020183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fcee7b-fe20-4202-87ad-60bc4d4a3c0d service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:24.032945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1077/2155] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:24.034004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcee7b-fe20-4202-87ad-60bc4d4a3c0d service nova] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:24.035612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-52fcee7b-fe20-4202-87ad-60bc4d4a3c0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:24.114012 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e119ab7-f4bf-4a58-bd0a-b21059446589 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:24.125170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:24.126193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:24.126537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:24.130717 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] PATH_INFO: `/v3/users/d22ffa619cbd426381c525736af882f6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:24.130717 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d22ffa619cbd426381c525736af882f6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:24.130717 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:24.183845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcee7b-fe20-4202-87ad-60bc4d4a3c0d service nova] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:24.193259 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d773f-0bf5-4a76-8d96-5d5c5ad8ca89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:24.193919 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d773f-0bf5-4a76-8d96-5d5c5ad8ca89 service swift] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:24.194204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d773f-0bf5-4a76-8d96-5d5c5ad8ca89 service swift] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:24.194479 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d773f-0bf5-4a76-8d96-5d5c5ad8ca89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:24.204257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d773f-0bf5-4a76-8d96-5d5c5ad8ca89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:24.206036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d773f-0bf5-4a76-8d96-5d5c5ad8ca89 service swift] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:24.225442 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1078/2156] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:01:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:01:24.289239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d773f-0bf5-4a76-8d96-5d5c5ad8ca89 service swift] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:24.290975 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3e6d773f-0bf5-4a76-8d96-5d5c5ad8ca89 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:24.304955 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d773f-0bf5-4a76-8d96-5d5c5ad8ca89 service swift] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:24.309856 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92321d-4d31-43e3-ad5c-0a1dbb26e385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:24.310728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92321d-4d31-43e3-ad5c-0a1dbb26e385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:24.311050 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92321d-4d31-43e3-ad5c-0a1dbb26e385 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:24.311321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92321d-4d31-43e3-ad5c-0a1dbb26e385 admin admin] PATH_INFO: `/v3/users/841c59c497444d23819c386349fc4e3d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:24.312493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92321d-4d31-43e3-ad5c-0a1dbb26e385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=841c59c497444d23819c386349fc4e3d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:24.314494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92321d-4d31-43e3-ad5c-0a1dbb26e385 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:24.403646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd22ffa619cbd426381c525736af882f6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.412578 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd22ffa619cbd426381c525736af882f6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.413298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd22ffa619cbd426381c525736af882f6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.414580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:24.433854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd22ffa619cbd426381c525736af882f6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.514297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd22ffa619cbd426381c525736af882f6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.521046 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1079/2157] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:23 2022] DELETE /identity/v3/users/d22ffa619cbd426381c525736af882f6 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:24.521950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:24.523415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:24.602306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebb54c-7cb8-41b0-811f-5f024960b48c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:24.607836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bd92321d-4d31-43e3-ad5c-0a1dbb26e385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '841c59c497444d23819c386349fc4e3d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.608853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:24.620451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:24.620451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:24.620451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] PATH_INFO: `/v3/projects/199bfffb12ca4b1b9b221a003f3df804` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:24.620451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] RBAC: Authorizing `identity:delete_project(project_id=199bfffb12ca4b1b9b221a003f3df804)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:24.620451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:24.621951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bd92321d-4d31-43e3-ad5c-0a1dbb26e385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '841c59c497444d23819c386349fc4e3d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.621951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bd92321d-4d31-43e3-ad5c-0a1dbb26e385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '841c59c497444d23819c386349fc4e3d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.631367 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bd92321d-4d31-43e3-ad5c-0a1dbb26e385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '841c59c497444d23819c386349fc4e3d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.699288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bd92321d-4d31-43e3-ad5c-0a1dbb26e385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '841c59c497444d23819c386349fc4e3d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.701048 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:24.706260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1079/2158] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:24 2022] DELETE /identity/v3/users/841c59c497444d23819c386349fc4e3d => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:01:24.707590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d1b78-a47d-4281-9859-7c45cae696e9 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:24.707873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d1b78-a47d-4281-9859-7c45cae696e9 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:24.708201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d1b78-a47d-4281-9859-7c45cae696e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:24.726074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-6e3d1b78-a47d-4281-9859-7c45cae696e9 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:24.727338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6e3d1b78-a47d-4281-9859-7c45cae696e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:24.750491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '199bfffb12ca4b1b9b221a003f3df804', 'request_id': 'req-6f34d9a0-543f-445c-abb9-a43fac268388'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.790894 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1080/2159] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:24.792790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb46eee-765f-4ce6-bb1c-4882058d10ac None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:24.793131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb46eee-765f-4ce6-bb1c-4882058d10ac None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:24.793457 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb46eee-765f-4ce6-bb1c-4882058d10ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:24.818244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-4bb46eee-765f-4ce6-bb1c-4882058d10ac None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:24.819498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4bb46eee-765f-4ce6-bb1c-4882058d10ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:24.864339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '199bfffb12ca4b1b9b221a003f3df804', 'request_id': 'req-6f34d9a0-543f-445c-abb9-a43fac268388'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.874680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '199bfffb12ca4b1b9b221a003f3df804', 'request_id': 'req-6f34d9a0-543f-445c-abb9-a43fac268388'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.887782 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] The token cache is being invalidate because project 199bfffb12ca4b1b9b221a003f3df804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:24.888495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.889608 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:24.891748 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1080/2160] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:24 2022] DELETE /identity/v3/projects/199bfffb12ca4b1b9b221a003f3df804 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:24.892788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:24.894439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:24.894894 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1081/2161] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:24.898623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23474716-da05-4d7a-85bd-51162e188380 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:24.899203 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23474716-da05-4d7a-85bd-51162e188380 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:24.899789 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23474716-da05-4d7a-85bd-51162e188380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:24.924703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-23474716-da05-4d7a-85bd-51162e188380 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:24.926181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-23474716-da05-4d7a-85bd-51162e188380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:24.945328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1082/2162] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:01:24.945328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23474716-da05-4d7a-85bd-51162e188380 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:24.945328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-23474716-da05-4d7a-85bd-51162e188380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:24.975816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34d9a0-543f-445c-abb9-a43fac268388 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:24.983539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106c072-f641-4fac-942e-c0f8a5b387d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:24.985150 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106c072-f641-4fac-942e-c0f8a5b387d1 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:24.985477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106c072-f641-4fac-942e-c0f8a5b387d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:24.985758 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106c072-f641-4fac-942e-c0f8a5b387d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:24.996639 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b106c072-f641-4fac-942e-c0f8a5b387d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:25.013855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106c072-f641-4fac-942e-c0f8a5b387d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:25.015800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106c072-f641-4fac-942e-c0f8a5b387d1 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:25.027770 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23474716-da05-4d7a-85bd-51162e188380 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:25.029762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1081/2163] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:25.031551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458aaa1-3763-4334-a576-0ac54fc2890f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.031962 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458aaa1-3763-4334-a576-0ac54fc2890f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.032246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458aaa1-3763-4334-a576-0ac54fc2890f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:25.032758 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b4078-b671-4f21-afa7-6e47010c24c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:25.033521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b4078-b671-4f21-afa7-6e47010c24c5 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.033978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b4078-b671-4f21-afa7-6e47010c24c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.034320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b4078-b671-4f21-afa7-6e47010c24c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:25.045976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b4078-b671-4f21-afa7-6e47010c24c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:25.048275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b4078-b671-4f21-afa7-6e47010c24c5 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:25.058193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7458aaa1-3763-4334-a576-0ac54fc2890f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:25.059787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7458aaa1-3763-4334-a576-0ac54fc2890f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:25.061608 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1083/2164] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:25.069391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1082/2165] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:25.073230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b4078-b671-4f21-afa7-6e47010c24c5 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:25.084602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b4078-b671-4f21-afa7-6e47010c24c5 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:25.090353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b75227-df8b-4c6b-bb2a-890f41520eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:25.096439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b75227-df8b-4c6b-bb2a-890f41520eac admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.096439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b75227-df8b-4c6b-bb2a-890f41520eac admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.096439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b75227-df8b-4c6b-bb2a-890f41520eac admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:25.096439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b75227-df8b-4c6b-bb2a-890f41520eac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:01:25.096439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b75227-df8b-4c6b-bb2a-890f41520eac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:25.096439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b75227-df8b-4c6b-bb2a-890f41520eac admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:25.109111 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1084/2166] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:01:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:01:25.113527 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458aaa1-3763-4334-a576-0ac54fc2890f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:25.126340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458aaa1-3763-4334-a576-0ac54fc2890f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:25.139427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff03d0d-de92-4e04-a4eb-099c7d78491f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:25.140209 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff03d0d-de92-4e04-a4eb-099c7d78491f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.140505 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff03d0d-de92-4e04-a4eb-099c7d78491f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.140782 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff03d0d-de92-4e04-a4eb-099c7d78491f admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:25.143247 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff03d0d-de92-4e04-a4eb-099c7d78491f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:25.145270 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff03d0d-de92-4e04-a4eb-099c7d78491f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:25.275850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1083/2167] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:01:25 2022] POST /identity/v3/projects => generated 374 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:01:25.282261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b75227-df8b-4c6b-bb2a-890f41520eac admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:25.292492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b75227-df8b-4c6b-bb2a-890f41520eac admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:25.349219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999db678-d626-4013-ab3d-3a0bad3a4ff2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.349523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999db678-d626-4013-ab3d-3a0bad3a4ff2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.349797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999db678-d626-4013-ab3d-3a0bad3a4ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:25.359279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459d94e-7491-4eee-b49b-254d051bd213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:25.359925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459d94e-7491-4eee-b49b-254d051bd213 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.360226 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459d94e-7491-4eee-b49b-254d051bd213 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.360472 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459d94e-7491-4eee-b49b-254d051bd213 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:25.361225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459d94e-7491-4eee-b49b-254d051bd213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:25.362635 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:25.363386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459d94e-7491-4eee-b49b-254d051bd213 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:25.381754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-999db678-d626-4013-ab3d-3a0bad3a4ff2 None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:25.383191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-999db678-d626-4013-ab3d-3a0bad3a4ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:25.438848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:25.456563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1084/2168] 10.222.0.121 () {58 vars in 972 bytes} [Thu May 26 18:01:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:01:25.472809 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999db678-d626-4013-ab3d-3a0bad3a4ff2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:25.474548 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-999db678-d626-4013-ab3d-3a0bad3a4ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:25.475200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1085/2169] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:01:25 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:25.480480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459d94e-7491-4eee-b49b-254d051bd213 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:25.503057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459d94e-7491-4eee-b49b-254d051bd213 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:25.509571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f032bcdb-e443-4258-bc3b-b65e2f6ee7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:25.511320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032bcdb-e443-4258-bc3b-b65e2f6ee7ad admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.511810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032bcdb-e443-4258-bc3b-b65e2f6ee7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.512206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032bcdb-e443-4258-bc3b-b65e2f6ee7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:25.513187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f032bcdb-e443-4258-bc3b-b65e2f6ee7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:25.515111 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f032bcdb-e443-4258-bc3b-b65e2f6ee7ad admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:25.527242 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1086/2170] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:25 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:25.531736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f032bcdb-e443-4258-bc3b-b65e2f6ee7ad admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:25.544249 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f032bcdb-e443-4258-bc3b-b65e2f6ee7ad admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:25.549835 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46330f9-c758-42ef-b9ac-73013da047f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:25.550671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46330f9-c758-42ef-b9ac-73013da047f7 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.551018 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46330f9-c758-42ef-b9ac-73013da047f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.551297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46330f9-c758-42ef-b9ac-73013da047f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:25.552672 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46330f9-c758-42ef-b9ac-73013da047f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:25.554452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46330f9-c758-42ef-b9ac-73013da047f7 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:25.571130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1087/2171] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:25 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:01:25.575332 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46330f9-c758-42ef-b9ac-73013da047f7 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:25.580162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999db678-d626-4013-ab3d-3a0bad3a4ff2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:25.587373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b29a0e-ffdc-4f17-9824-9225eac7e18d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:25.590100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46330f9-c758-42ef-b9ac-73013da047f7 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:25.590332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b29a0e-ffdc-4f17-9824-9225eac7e18d service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.590332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b29a0e-ffdc-4f17-9824-9225eac7e18d service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.590332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b29a0e-ffdc-4f17-9824-9225eac7e18d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:25.594428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f848e3-822d-4aaf-a8fe-ecab9b457991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:25.595365 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f848e3-822d-4aaf-a8fe-ecab9b457991 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.595643 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f848e3-822d-4aaf-a8fe-ecab9b457991 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.595975 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f848e3-822d-4aaf-a8fe-ecab9b457991 admin admin] PATH_INFO: `/v3/projects/9772006524234da68b7c89ba9355c8ec/users/848f842622044dd284a310c19c6b0dac/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:25.600251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b29a0e-ffdc-4f17-9824-9225eac7e18d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:25.601933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b29a0e-ffdc-4f17-9824-9225eac7e18d service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:25.615280 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1085/2172] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:01:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:01:25.615864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f848e3-822d-4aaf-a8fe-ecab9b457991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9772006524234da68b7c89ba9355c8ec, user_id=848f842622044dd284a310c19c6b0dac, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:25.617827 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f848e3-822d-4aaf-a8fe-ecab9b457991 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:25.643112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b29a0e-ffdc-4f17-9824-9225eac7e18d service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:25.644615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d0b29a0e-ffdc-4f17-9824-9225eac7e18d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:25.661845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b29a0e-ffdc-4f17-9824-9225eac7e18d service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:25.668021 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:25.669079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.669442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.669946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20 admin admin] PATH_INFO: `/v3/projects/a7cf21850a5d4df7b7e3d145c0c2402b` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:25.671769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7cf21850a5d4df7b7e3d145c0c2402b)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:25.674210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:25.738809 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1088/2173] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:25 2022] PUT /identity/v3/projects/9772006524234da68b7c89ba9355c8ec/users/848f842622044dd284a310c19c6b0dac/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:25.739918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f848e3-822d-4aaf-a8fe-ecab9b457991 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:25.744425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-06f848e3-822d-4aaf-a8fe-ecab9b457991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:25.821196 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f848e3-822d-4aaf-a8fe-ecab9b457991 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:25.839997 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7cf21850a5d4df7b7e3d145c0c2402b', 'request_id': 'req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:25.882720 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:25.883608 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.883885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.884154 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] PATH_INFO: `/v3/projects/588c313eba2e4469bc1206ab2794a2c2` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:25.885344 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=588c313eba2e4469bc1206ab2794a2c2)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:25.887341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:25.903700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:25.948583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7cf21850a5d4df7b7e3d145c0c2402b', 'request_id': 'req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:25.955554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7cf21850a5d4df7b7e3d145c0c2402b', 'request_id': 'req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:25.960594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:25.969872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20 admin admin] The token cache is being invalidate because project a7cf21850a5d4df7b7e3d145c0c2402b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:25.970379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:25.971070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4551acf6-d8e0-4c25-871a-9ec1a2c0ae20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:25.972892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1086/2174] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:25 2022] DELETE /identity/v3/projects/a7cf21850a5d4df7b7e3d145c0c2402b => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:25.976177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86217a-98bd-425c-aad7-c6844ce52ccf None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:25.976177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86217a-98bd-425c-aad7-c6844ce52ccf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:25.976177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86217a-98bd-425c-aad7-c6844ce52ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.005334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4b86217a-98bd-425c-aad7-c6844ce52ccf None None] MFA Rules not processed for user `848f842622044dd284a310c19c6b0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:26.006627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4b86217a-98bd-425c-aad7-c6844ce52ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:26.029141 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '588c313eba2e4469bc1206ab2794a2c2', 'request_id': 'req-d7d73c6e-9af9-4089-8c87-de960ae93b20'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:26.095040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1087/2175] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:26.095040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb74f9f-aeb1-4a6c-bc96-37030e129e14 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.095040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb74f9f-aeb1-4a6c-bc96-37030e129e14 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.095040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb74f9f-aeb1-4a6c-bc96-37030e129e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.108183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '588c313eba2e4469bc1206ab2794a2c2', 'request_id': 'req-d7d73c6e-9af9-4089-8c87-de960ae93b20'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:26.117050 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '588c313eba2e4469bc1206ab2794a2c2', 'request_id': 'req-d7d73c6e-9af9-4089-8c87-de960ae93b20'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:26.128233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-fdb74f9f-aeb1-4a6c-bc96-37030e129e14 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:26.129443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fdb74f9f-aeb1-4a6c-bc96-37030e129e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:26.131964 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] The token cache is being invalidate because project 588c313eba2e4469bc1206ab2794a2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:26.138331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:26.138331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:26.138331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1089/2176] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:25 2022] DELETE /identity/v3/projects/588c313eba2e4469bc1206ab2794a2c2 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:26.138331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:26.140540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:26.197809 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1088/2177] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:26.199963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b56fd8-53ff-44dc-8252-30725ac31080 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.200380 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b56fd8-53ff-44dc-8252-30725ac31080 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.200780 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b56fd8-53ff-44dc-8252-30725ac31080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.219930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-74b56fd8-53ff-44dc-8252-30725ac31080 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:26.221277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-74b56fd8-53ff-44dc-8252-30725ac31080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:26.237092 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1089/2178] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:26.244771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59261f-cc7c-4556-86aa-b2f671214810 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.244771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59261f-cc7c-4556-86aa-b2f671214810 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.251518 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d73c6e-9af9-4089-8c87-de960ae93b20 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:26.251616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59261f-cc7c-4556-86aa-b2f671214810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.251700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571a8c9-edd3-4675-a4e7-6e17485b0a73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:26.251700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571a8c9-edd3-4675-a4e7-6e17485b0a73 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.251700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571a8c9-edd3-4675-a4e7-6e17485b0a73 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.251700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571a8c9-edd3-4675-a4e7-6e17485b0a73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.252267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0571a8c9-edd3-4675-a4e7-6e17485b0a73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:26.263343 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7d59261f-cc7c-4556-86aa-b2f671214810 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:26.263704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7d59261f-cc7c-4556-86aa-b2f671214810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:26.277502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1090/2179] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:26.277502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b1a31-c613-4ab6-87d4-3e66ba897fbb None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.277502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b1a31-c613-4ab6-87d4-3e66ba897fbb None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.277502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b1a31-c613-4ab6-87d4-3e66ba897fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.300170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-562b1a31-c613-4ab6-87d4-3e66ba897fbb None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:26.302393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-562b1a31-c613-4ab6-87d4-3e66ba897fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:26.313005 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1091/2180] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:26.313758 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b1a31-c613-4ab6-87d4-3e66ba897fbb None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:26.325942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b1a31-c613-4ab6-87d4-3e66ba897fbb None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:26.333503 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51565e80-eb83-47ce-beff-a65bda7365d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:26.335102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51565e80-eb83-47ce-beff-a65bda7365d6 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.335811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51565e80-eb83-47ce-beff-a65bda7365d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.336751 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51565e80-eb83-47ce-beff-a65bda7365d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.338318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51565e80-eb83-47ce-beff-a65bda7365d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:01:26.339433 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51565e80-eb83-47ce-beff-a65bda7365d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:26.341678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51565e80-eb83-47ce-beff-a65bda7365d6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:26.343367 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571a8c9-edd3-4675-a4e7-6e17485b0a73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:26.348849 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571a8c9-edd3-4675-a4e7-6e17485b0a73 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:26.355827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1092/2181] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:01:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:01:26.356809 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51565e80-eb83-47ce-beff-a65bda7365d6 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:26.360002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1090/2182] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:26.360959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571a8c9-edd3-4675-a4e7-6e17485b0a73 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:26.376189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51565e80-eb83-47ce-beff-a65bda7365d6 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:26.379837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571a8c9-edd3-4675-a4e7-6e17485b0a73 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:26.386911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc18a2-8822-46b0-9fb5-c4941b5d5b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:26.387974 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc18a2-8822-46b0-9fb5-c4941b5d5b24 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.388259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc18a2-8822-46b0-9fb5-c4941b5d5b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.388586 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc18a2-8822-46b0-9fb5-c4941b5d5b24 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.389266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc18a2-8822-46b0-9fb5-c4941b5d5b24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:01:26.389627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc18a2-8822-46b0-9fb5-c4941b5d5b24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:26.391369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc18a2-8822-46b0-9fb5-c4941b5d5b24 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:26.391498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcfb04-14eb-4d80-a521-544ac32a4db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:26.392244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcfb04-14eb-4d80-a521-544ac32a4db0 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.392594 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcfb04-14eb-4d80-a521-544ac32a4db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.393111 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcfb04-14eb-4d80-a521-544ac32a4db0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.393808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcfb04-14eb-4d80-a521-544ac32a4db0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:26.396026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcfb04-14eb-4d80-a521-544ac32a4db0 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:26.406508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1093/2183] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:01:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:01:26.407769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc18a2-8822-46b0-9fb5-c4941b5d5b24 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:26.409530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-44dc18a2-8822-46b0-9fb5-c4941b5d5b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:26.427090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc18a2-8822-46b0-9fb5-c4941b5d5b24 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:26.435713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:26.436578 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.436858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.437191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] PATH_INFO: `/v3/users/848f842622044dd284a310c19c6b0dac` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.438510 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=848f842622044dd284a310c19c6b0dac)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:26.440733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:26.496829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1091/2184] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:01:26 2022] POST /identity/v3/projects => generated 374 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:26.496829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcfb04-14eb-4d80-a521-544ac32a4db0 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:26.521801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:26.578510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcfb04-14eb-4d80-a521-544ac32a4db0 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:26.589845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29402494-4c5f-4959-bf25-085dfcc7c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:26.590681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29402494-4c5f-4959-bf25-085dfcc7c6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.591003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29402494-4c5f-4959-bf25-085dfcc7c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.591278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29402494-4c5f-4959-bf25-085dfcc7c6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.592022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29402494-4c5f-4959-bf25-085dfcc7c6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:26.593998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29402494-4c5f-4959-bf25-085dfcc7c6d1 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:26.648073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:26.657317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '848f842622044dd284a310c19c6b0dac'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:26.664598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '848f842622044dd284a310c19c6b0dac'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:26.665297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '848f842622044dd284a310c19c6b0dac'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:26.678436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '848f842622044dd284a310c19c6b0dac'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:26.681677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1092/2185] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:01:26 2022] POST /identity/v3/projects => generated 366 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:26.684710 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83b240-dabc-4c58-8844-04070c7917dd None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.685200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83b240-dabc-4c58-8844-04070c7917dd None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.685683 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83b240-dabc-4c58-8844-04070c7917dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.706377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3d83b240-dabc-4c58-8844-04070c7917dd None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:26.707661 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3d83b240-dabc-4c58-8844-04070c7917dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:26.765540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1093/2186] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 18:01:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:26.766465 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83b240-dabc-4c58-8844-04070c7917dd None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:26.771374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '848f842622044dd284a310c19c6b0dac'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:26.780592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1094/2187] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:26 2022] DELETE /identity/v3/users/848f842622044dd284a310c19c6b0dac => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:26.782096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:26.783786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:26.841104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83b240-dabc-4c58-8844-04070c7917dd None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:26.858219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a464c-04d7-4d57-a940-9bf97d307821 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:26.864666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370f0db-fde6-4a9d-98cc-396c64b3c23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:26.865339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370f0db-fde6-4a9d-98cc-396c64b3c23c admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.865620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370f0db-fde6-4a9d-98cc-396c64b3c23c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.865896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370f0db-fde6-4a9d-98cc-396c64b3c23c admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.867030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370f0db-fde6-4a9d-98cc-396c64b3c23c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:26.869354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370f0db-fde6-4a9d-98cc-396c64b3c23c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:26.890628 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:26.900458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25686f14-7e08-4852-ae50-249bf93d2e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:26.901674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25686f14-7e08-4852-ae50-249bf93d2e41 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:26.902144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25686f14-7e08-4852-ae50-249bf93d2e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:26.902635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25686f14-7e08-4852-ae50-249bf93d2e41 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:26.903437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25686f14-7e08-4852-ae50-249bf93d2e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:26.905520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25686f14-7e08-4852-ae50-249bf93d2e41 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.037364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1095/2188] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:01:26 2022] POST /identity/v3/users => generated 284 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:27.038229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370f0db-fde6-4a9d-98cc-396c64b3c23c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.039827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7370f0db-fde6-4a9d-98cc-396c64b3c23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:27.057878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1094/2189] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:01:26 2022] POST /identity/v3/users => generated 288 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:27.059540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f6d6a-f27e-4ddc-903b-776ac4172df6 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.059829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f6d6a-f27e-4ddc-903b-776ac4172df6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.060100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f6d6a-f27e-4ddc-903b-776ac4172df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.079661 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-413f6d6a-f27e-4ddc-903b-776ac4172df6 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:27.080888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-413f6d6a-f27e-4ddc-903b-776ac4172df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:27.090640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1095/2190] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:27.091333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f6d6a-f27e-4ddc-903b-776ac4172df6 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.109130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f6d6a-f27e-4ddc-903b-776ac4172df6 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.115455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb7d2f-15a0-492a-92fd-bce9b3cc88cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.116423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb7d2f-15a0-492a-92fd-bce9b3cc88cf admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.116871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb7d2f-15a0-492a-92fd-bce9b3cc88cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.117318 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb7d2f-15a0-492a-92fd-bce9b3cc88cf admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.118658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb7d2f-15a0-492a-92fd-bce9b3cc88cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.121339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb7d2f-15a0-492a-92fd-bce9b3cc88cf admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.122718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370f0db-fde6-4a9d-98cc-396c64b3c23c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.128205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b5b32-5f30-46da-a60f-c348dd445285 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.128864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b5b32-5f30-46da-a60f-c348dd445285 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.129144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b5b32-5f30-46da-a60f-c348dd445285 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.129684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b5b32-5f30-46da-a60f-c348dd445285 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.131291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-881b5b32-5f30-46da-a60f-c348dd445285 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:27.138199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1096/2191] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:27 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:27.139130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb7d2f-15a0-492a-92fd-bce9b3cc88cf admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.140749 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4afb7d2f-15a0-492a-92fd-bce9b3cc88cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:27.154567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb7d2f-15a0-492a-92fd-bce9b3cc88cf admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.159468 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7535e1-9d10-41f5-9780-95b1012d20da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.160138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7535e1-9d10-41f5-9780-95b1012d20da admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.160421 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7535e1-9d10-41f5-9780-95b1012d20da admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.160696 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7535e1-9d10-41f5-9780-95b1012d20da admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.161424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7535e1-9d10-41f5-9780-95b1012d20da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.163171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7535e1-9d10-41f5-9780-95b1012d20da admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.179884 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1097/2192] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:27 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:27.180741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7535e1-9d10-41f5-9780-95b1012d20da admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.182655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0c7535e1-9d10-41f5-9780-95b1012d20da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:27.183518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:27.197674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b5b32-5f30-46da-a60f-c348dd445285 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.200014 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b5b32-5f30-46da-a60f-c348dd445285 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.208731 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1096/2193] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:01:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:01:27.209566 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b5b32-5f30-46da-a60f-c348dd445285 service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.221557 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b5b32-5f30-46da-a60f-c348dd445285 service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.226740 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208afd4-a66d-40ef-8843-6cabdb35f9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.227552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208afd4-a66d-40ef-8843-6cabdb35f9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.227841 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208afd4-a66d-40ef-8843-6cabdb35f9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.228151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208afd4-a66d-40ef-8843-6cabdb35f9d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.228959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208afd4-a66d-40ef-8843-6cabdb35f9d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.230857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208afd4-a66d-40ef-8843-6cabdb35f9d5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.242105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1097/2194] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:27 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:27.243575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208afd4-a66d-40ef-8843-6cabdb35f9d5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.247389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7535e1-9d10-41f5-9780-95b1012d20da admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.251285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcad823-e536-40b7-86d8-4a089ddf49f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.254677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcad823-e536-40b7-86d8-4a089ddf49f4 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.254677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcad823-e536-40b7-86d8-4a089ddf49f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.254677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcad823-e536-40b7-86d8-4a089ddf49f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.255062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208afd4-a66d-40ef-8843-6cabdb35f9d5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.259820 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb66c6b-9ca6-4c78-a609-bb6e03bce1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.261055 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb66c6b-9ca6-4c78-a609-bb6e03bce1dd admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.261287 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb66c6b-9ca6-4c78-a609-bb6e03bce1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.261549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb66c6b-9ca6-4c78-a609-bb6e03bce1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.262464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb66c6b-9ca6-4c78-a609-bb6e03bce1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.264100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcad823-e536-40b7-86d8-4a089ddf49f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.264257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb66c6b-9ca6-4c78-a609-bb6e03bce1dd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.265523 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcad823-e536-40b7-86d8-4a089ddf49f4 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.275560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1098/2195] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:27 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:27.276693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1098/2196] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:27.277595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb66c6b-9ca6-4c78-a609-bb6e03bce1dd admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.280398 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcad823-e536-40b7-86d8-4a089ddf49f4 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.288942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb66c6b-9ca6-4c78-a609-bb6e03bce1dd admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.291533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcad823-e536-40b7-86d8-4a089ddf49f4 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.294692 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80427371-f2f0-4ffe-9d27-34debef41fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.295770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80427371-f2f0-4ffe-9d27-34debef41fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.301165 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80427371-f2f0-4ffe-9d27-34debef41fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.301165 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80427371-f2f0-4ffe-9d27-34debef41fe5 admin admin] PATH_INFO: `/v3/projects/8b062a27c4c244ef9107cd2027269009/users/a176a96e7e4e483ea1aeeb9e6ee44239/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.301407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b03fed-4250-44b9-b6e5-f4fdbe073c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.301407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b03fed-4250-44b9-b6e5-f4fdbe073c3a admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.301407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b03fed-4250-44b9-b6e5-f4fdbe073c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.301407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b03fed-4250-44b9-b6e5-f4fdbe073c3a admin admin] PATH_INFO: `/v3/projects/e358a1e391cf4ea18f18b10ed3585a91/users/9dc1b321e70f42e2be177e161486f997/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.314052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80427371-f2f0-4ffe-9d27-34debef41fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b062a27c4c244ef9107cd2027269009, user_id=a176a96e7e4e483ea1aeeb9e6ee44239, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.315772 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b03fed-4250-44b9-b6e5-f4fdbe073c3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e358a1e391cf4ea18f18b10ed3585a91, user_id=9dc1b321e70f42e2be177e161486f997, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.316787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80427371-f2f0-4ffe-9d27-34debef41fe5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.318888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b03fed-4250-44b9-b6e5-f4fdbe073c3a admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.406735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1099/2197] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:27 2022] PUT /identity/v3/projects/8b062a27c4c244ef9107cd2027269009/users/a176a96e7e4e483ea1aeeb9e6ee44239/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:27.421579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c833-2172-47e4-8a44-f03ee09a1156 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.421870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c833-2172-47e4-8a44-f03ee09a1156 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.422140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c833-2172-47e4-8a44-f03ee09a1156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.450269 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e9e6c833-2172-47e4-8a44-f03ee09a1156 None None] MFA Rules not processed for user `a176a96e7e4e483ea1aeeb9e6ee44239`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:27.451531 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e9e6c833-2172-47e4-8a44-f03ee09a1156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:27.480449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1099/2198] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:27 2022] PUT /identity/v3/projects/e358a1e391cf4ea18f18b10ed3585a91/users/9dc1b321e70f42e2be177e161486f997/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:01:27.485514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ba482-a01d-4ed5-b988-84e15bc6bd38 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.485801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ba482-a01d-4ed5-b988-84e15bc6bd38 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.486082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ba482-a01d-4ed5-b988-84e15bc6bd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.505044 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-549ba482-a01d-4ed5-b988-84e15bc6bd38 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:27.506262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-549ba482-a01d-4ed5-b988-84e15bc6bd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:27.525869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1100/2199] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:01:27.526931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6c833-2172-47e4-8a44-f03ee09a1156 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.592336 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1100/2200] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:01:27.598514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ba482-a01d-4ed5-b988-84e15bc6bd38 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.612025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6c833-2172-47e4-8a44-f03ee09a1156 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.618521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fe02a-fcff-45af-9cb2-263c8061a922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.621118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fe02a-fcff-45af-9cb2-263c8061a922 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.621118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fe02a-fcff-45af-9cb2-263c8061a922 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.621118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fe02a-fcff-45af-9cb2-263c8061a922 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.621118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fe02a-fcff-45af-9cb2-263c8061a922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.623200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fe02a-fcff-45af-9cb2-263c8061a922 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.662029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:27.689088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ba482-a01d-4ed5-b988-84e15bc6bd38 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.692884 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd731db-f116-4bba-9240-39429a474ad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.693546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd731db-f116-4bba-9240-39429a474ad5 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.693826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd731db-f116-4bba-9240-39429a474ad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.694097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd731db-f116-4bba-9240-39429a474ad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.707747 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd731db-f116-4bba-9240-39429a474ad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.709382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd731db-f116-4bba-9240-39429a474ad5 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.722123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1101/2201] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:27.726376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1101/2202] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:01:27 2022] POST /identity/v3/projects => generated 366 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:27.732756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd731db-f116-4bba-9240-39429a474ad5 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.747624 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd731db-f116-4bba-9240-39429a474ad5 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.820454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8154eb9-9d20-43c0-9fea-e26169b1e601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.821502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8154eb9-9d20-43c0-9fea-e26169b1e601 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.821866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8154eb9-9d20-43c0-9fea-e26169b1e601 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.822211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8154eb9-9d20-43c0-9fea-e26169b1e601 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.823664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8154eb9-9d20-43c0-9fea-e26169b1e601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.825571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8154eb9-9d20-43c0-9fea-e26169b1e601 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.927157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1102/2203] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:01:27 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:01:27.930043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fe02a-fcff-45af-9cb2-263c8061a922 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.931262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8154eb9-9d20-43c0-9fea-e26169b1e601 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:27.931896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-680fe02a-fcff-45af-9cb2-263c8061a922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:27.944880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8154eb9-9d20-43c0-9fea-e26169b1e601 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.953387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fe02a-fcff-45af-9cb2-263c8061a922 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:27.958918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eab37e-e0e5-4edf-9a1f-1111a505e10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.961117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eab37e-e0e5-4edf-9a1f-1111a505e10b admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.961117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eab37e-e0e5-4edf-9a1f-1111a505e10b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.961117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eab37e-e0e5-4edf-9a1f-1111a505e10b admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.963202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eab37e-e0e5-4edf-9a1f-1111a505e10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.965689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eab37e-e0e5-4edf-9a1f-1111a505e10b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.975141 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:27.976177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.978171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.978171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] PATH_INFO: `/v3/projects/9772006524234da68b7c89ba9355c8ec` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.978171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=9772006524234da68b7c89ba9355c8ec)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:27.981780 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:27.987519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1103/2204] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:27 2022] GET /identity/v3/roles => generated 1427 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:01:27.987519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911130c8-bc51-4eee-a541-0a00b25f0d43 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:27.987519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911130c8-bc51-4eee-a541-0a00b25f0d43 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:27.987519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911130c8-bc51-4eee-a541-0a00b25f0d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:27.992712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:28.011780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-911130c8-bc51-4eee-a541-0a00b25f0d43 None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:28.013155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-911130c8-bc51-4eee-a541-0a00b25f0d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:28.072866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1104/2205] 10.222.0.121 () {58 vars in 972 bytes} [Thu May 26 18:01:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:28.074313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911130c8-bc51-4eee-a541-0a00b25f0d43 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:28.086297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911130c8-bc51-4eee-a541-0a00b25f0d43 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:28.093002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906afb9-0610-4622-b96e-6e6fac92022c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:28.093913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906afb9-0610-4622-b96e-6e6fac92022c admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:28.094193 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906afb9-0610-4622-b96e-6e6fac92022c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:28.094471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906afb9-0610-4622-b96e-6e6fac92022c admin admin] PATH_INFO: `/v3/projects/8e7feaab64dc42cc85fff5f23b37d442/users/9c54c05453294e7093430c6055cbefaf/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:28.110244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906afb9-0610-4622-b96e-6e6fac92022c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e7feaab64dc42cc85fff5f23b37d442, user_id=9c54c05453294e7093430c6055cbefaf, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:28.112945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906afb9-0610-4622-b96e-6e6fac92022c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:28.199959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1105/2206] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:28 2022] PUT /identity/v3/projects/8e7feaab64dc42cc85fff5f23b37d442/users/9c54c05453294e7093430c6055cbefaf/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:28.204638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906afb9-0610-4622-b96e-6e6fac92022c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:28.217013 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9772006524234da68b7c89ba9355c8ec', 'request_id': 'req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:28.256069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:28.285359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906afb9-0610-4622-b96e-6e6fac92022c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:28.291035 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9772006524234da68b7c89ba9355c8ec', 'request_id': 'req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:28.299009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9772006524234da68b7c89ba9355c8ec', 'request_id': 'req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:28.310720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] The token cache is being invalidate because project 9772006524234da68b7c89ba9355c8ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:28.311063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:28.311714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:28.313815 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1102/2207] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:27 2022] DELETE /identity/v3/projects/9772006524234da68b7c89ba9355c8ec => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:28.314603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:28.315994 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:28.357182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53a90e-8da4-4a4b-9d15-3f0ea63ccf0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:28.357989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53a90e-8da4-4a4b-9d15-3f0ea63ccf0a service swift] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:28.358279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53a90e-8da4-4a4b-9d15-3f0ea63ccf0a service swift] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:28.358585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53a90e-8da4-4a4b-9d15-3f0ea63ccf0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:28.360724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-fc53a90e-8da4-4a4b-9d15-3f0ea63ccf0a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:28.389166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4cbd8a-3108-45b3-bcfd-85d6965b63ae admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:28.394374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5d1bf-6260-40d1-aef3-2375b6764ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:28.395180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5d1bf-6260-40d1-aef3-2375b6764ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:28.395526 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5d1bf-6260-40d1-aef3-2375b6764ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:28.395810 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5d1bf-6260-40d1-aef3-2375b6764ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:28.396555 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5d1bf-6260-40d1-aef3-2375b6764ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:28.398192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5d1bf-6260-40d1-aef3-2375b6764ea3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:28.412552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1103/2208] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:28 2022] GET /identity/v3/roles => generated 1427 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:28.414072 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e63daa-1a43-4f24-825f-7937a035ed1f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:28.414536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e63daa-1a43-4f24-825f-7937a035ed1f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:28.414952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e63daa-1a43-4f24-825f-7937a035ed1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:28.438995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-33e63daa-1a43-4f24-825f-7937a035ed1f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:28.441139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-33e63daa-1a43-4f24-825f-7937a035ed1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:28.448866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53a90e-8da4-4a4b-9d15-3f0ea63ccf0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:28.450721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53a90e-8da4-4a4b-9d15-3f0ea63ccf0a service swift] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:28.463613 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1104/2209] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:28.463613 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e63daa-1a43-4f24-825f-7937a035ed1f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:28.469442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e63daa-1a43-4f24-825f-7937a035ed1f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:28.469607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1106/2210] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:01:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:01:28.471362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901ae42-8346-4db2-af2d-7e769238a4f5 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:28.471840 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901ae42-8346-4db2-af2d-7e769238a4f5 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:28.472396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901ae42-8346-4db2-af2d-7e769238a4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:28.475496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a2c90-ff8e-473d-8c0a-428d1ba3f34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:28.476411 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2c90-ff8e-473d-8c0a-428d1ba3f34f admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:28.476691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2c90-ff8e-473d-8c0a-428d1ba3f34f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:28.476963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2c90-ff8e-473d-8c0a-428d1ba3f34f admin admin] PATH_INFO: `/v3/projects/8e7feaab64dc42cc85fff5f23b37d442/users/9c54c05453294e7093430c6055cbefaf/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:28.489081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a2c90-ff8e-473d-8c0a-428d1ba3f34f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e7feaab64dc42cc85fff5f23b37d442, user_id=9c54c05453294e7093430c6055cbefaf, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:28.492021 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a2c90-ff8e-473d-8c0a-428d1ba3f34f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:28.506442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1105/2211] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:28 2022] PUT /identity/v3/projects/8e7feaab64dc42cc85fff5f23b37d442/users/9c54c05453294e7093430c6055cbefaf/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:28.517749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b828ff-d72b-43a6-a84a-f660ba8ad8fe None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:28.518185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b828ff-d72b-43a6-a84a-f660ba8ad8fe None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:28.518553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b828ff-d72b-43a6-a84a-f660ba8ad8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:28.519543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b901ae42-8346-4db2-af2d-7e769238a4f5 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:28.520785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b901ae42-8346-4db2-af2d-7e769238a4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:28.554630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e6b828ff-d72b-43a6-a84a-f660ba8ad8fe None None] MFA Rules not processed for user `9c54c05453294e7093430c6055cbefaf`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:28.555894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e6b828ff-d72b-43a6-a84a-f660ba8ad8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:28.605104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1107/2212] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:28.610136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901ae42-8346-4db2-af2d-7e769238a4f5 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:28.620787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901ae42-8346-4db2-af2d-7e769238a4f5 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:28.628276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d571c3-b8f6-49a0-bbb0-e169682984c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:28.628276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d571c3-b8f6-49a0-bbb0-e169682984c9 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:28.628276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d571c3-b8f6-49a0-bbb0-e169682984c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:28.628276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d571c3-b8f6-49a0-bbb0-e169682984c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:28.628674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d571c3-b8f6-49a0-bbb0-e169682984c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:01:28.629004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d571c3-b8f6-49a0-bbb0-e169682984c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:28.630607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d571c3-b8f6-49a0-bbb0-e169682984c9 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:28.645547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1108/2213] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:01:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:01:28.650085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d571c3-b8f6-49a0-bbb0-e169682984c9 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:28.653906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:28.664184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d571c3-b8f6-49a0-bbb0-e169682984c9 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:28.669884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1106/2214] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:28.671179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845c55d-30a5-4eed-bdbf-b9034b0b42ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:28.672786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845c55d-30a5-4eed-bdbf-b9034b0b42ab admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:28.673229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845c55d-30a5-4eed-bdbf-b9034b0b42ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:28.673682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845c55d-30a5-4eed-bdbf-b9034b0b42ab admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:28.674687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845c55d-30a5-4eed-bdbf-b9034b0b42ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:28.677236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845c55d-30a5-4eed-bdbf-b9034b0b42ab admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:28.681476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b828ff-d72b-43a6-a84a-f660ba8ad8fe None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:28.681644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:28.684090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e6b828ff-d72b-43a6-a84a-f660ba8ad8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:28.752066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b828ff-d72b-43a6-a84a-f660ba8ad8fe None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:28.758291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ac11e-3abd-4e08-954a-e275ca14f803 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:28.759696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ac11e-3abd-4e08-954a-e275ca14f803 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:28.760291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ac11e-3abd-4e08-954a-e275ca14f803 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:28.760641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ac11e-3abd-4e08-954a-e275ca14f803 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:28.761304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1109/2215] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:01:28 2022] POST /identity/v3/projects => generated 362 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:01:28.766846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845c55d-30a5-4eed-bdbf-b9034b0b42ab admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:28.774322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ac11e-3abd-4e08-954a-e275ca14f803 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:28.777361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ac11e-3abd-4e08-954a-e275ca14f803 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:28.794525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1107/2216] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:01:28.855613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845c55d-30a5-4eed-bdbf-b9034b0b42ab admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:28.863807 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626b325-398e-4b05-be69-b60a3cb5b41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:28.865130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626b325-398e-4b05-be69-b60a3cb5b41b admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:28.865130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626b325-398e-4b05-be69-b60a3cb5b41b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:28.865313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626b325-398e-4b05-be69-b60a3cb5b41b admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:28.866017 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626b325-398e-4b05-be69-b60a3cb5b41b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:28.868278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626b325-398e-4b05-be69-b60a3cb5b41b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:28.963282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1110/2217] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:01:28 2022] POST /identity/v3/users => generated 282 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:28.967559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ac11e-3abd-4e08-954a-e275ca14f803 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:28.988620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ac11e-3abd-4e08-954a-e275ca14f803 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:29.003239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6baf4-6beb-402a-9237-9b544fd3c6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:29.003921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6baf4-6beb-402a-9237-9b544fd3c6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:29.004199 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6baf4-6beb-402a-9237-9b544fd3c6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:29.004539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6baf4-6beb-402a-9237-9b544fd3c6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:29.007726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6baf4-6beb-402a-9237-9b544fd3c6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:29.009579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6baf4-6beb-402a-9237-9b544fd3c6e0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:29.016351 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626b325-398e-4b05-be69-b60a3cb5b41b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:29.025251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1108/2218] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:28 2022] GET /identity/v3/roles => generated 1427 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:29.030074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6baf4-6beb-402a-9237-9b544fd3c6e0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:29.051844 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6baf4-6beb-402a-9237-9b544fd3c6e0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:29.060989 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861e243-28e8-466c-ab75-dab96bc2ad71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:29.061726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861e243-28e8-466c-ab75-dab96bc2ad71 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:29.062025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861e243-28e8-466c-ab75-dab96bc2ad71 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:29.063103 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861e243-28e8-466c-ab75-dab96bc2ad71 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:29.063708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861e243-28e8-466c-ab75-dab96bc2ad71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:29.065760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861e243-28e8-466c-ab75-dab96bc2ad71 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:29.080440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1109/2219] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:29 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:01:29.087370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861e243-28e8-466c-ab75-dab96bc2ad71 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:29.098222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861e243-28e8-466c-ab75-dab96bc2ad71 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:29.108164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf3526-0096-4689-9218-9f0ef2877224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:29.109110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf3526-0096-4689-9218-9f0ef2877224 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:29.112954 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf3526-0096-4689-9218-9f0ef2877224 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:29.112954 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf3526-0096-4689-9218-9f0ef2877224 admin admin] PATH_INFO: `/v3/projects/39ca39bc5f714883b141c96b65d6e826/users/d15d5cc17aff4fd885ee56e8eee296c4/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:29.118489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626b325-398e-4b05-be69-b60a3cb5b41b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:29.124866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ddb94-cbaf-4e22-93a1-f0d1b1c9442d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:29.125544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ddb94-cbaf-4e22-93a1-f0d1b1c9442d service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:29.126040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ddb94-cbaf-4e22-93a1-f0d1b1c9442d service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:29.126172 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ddb94-cbaf-4e22-93a1-f0d1b1c9442d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:29.128772 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-608ddb94-cbaf-4e22-93a1-f0d1b1c9442d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:29.129847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf3526-0096-4689-9218-9f0ef2877224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ca39bc5f714883b141c96b65d6e826, user_id=d15d5cc17aff4fd885ee56e8eee296c4, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:29.132560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf3526-0096-4689-9218-9f0ef2877224 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:29.196158 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ddb94-cbaf-4e22-93a1-f0d1b1c9442d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:29.197509 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ddb94-cbaf-4e22-93a1-f0d1b1c9442d service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:29.207852 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1111/2220] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:01:29.228243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1110/2221] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:29 2022] PUT /identity/v3/projects/39ca39bc5f714883b141c96b65d6e826/users/d15d5cc17aff4fd885ee56e8eee296c4/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:01:29.230812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb456903-0d8c-4353-b8bf-1f22888daaba None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:29.231127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb456903-0d8c-4353-b8bf-1f22888daaba None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:29.231577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb456903-0d8c-4353-b8bf-1f22888daaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:29.270313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-cb456903-0d8c-4353-b8bf-1f22888daaba None None] MFA Rules not processed for user `d15d5cc17aff4fd885ee56e8eee296c4`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:29.271780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-cb456903-0d8c-4353-b8bf-1f22888daaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:29.290436 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:29.329161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa559f8-5200-4a8c-bb59-dcc21cada2e4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:29.329705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa559f8-5200-4a8c-bb59-dcc21cada2e4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:29.330164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa559f8-5200-4a8c-bb59-dcc21cada2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:29.370174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8fa559f8-5200-4a8c-bb59-dcc21cada2e4 None None] MFA Rules not processed for user `9dc1b321e70f42e2be177e161486f997`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:29.370355 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1112/2222] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:29.375852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8fa559f8-5200-4a8c-bb59-dcc21cada2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:29.463457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1111/2223] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:01:29.601227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb456903-0d8c-4353-b8bf-1f22888daaba None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:29.605550 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-cb456903-0d8c-4353-b8bf-1f22888daaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:29.660205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb456903-0d8c-4353-b8bf-1f22888daaba None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:29.663737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7aed3e-3f5b-43b9-ba6c-c33ea41dc28a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:29.664382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7aed3e-3f5b-43b9-ba6c-c33ea41dc28a service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:29.664671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7aed3e-3f5b-43b9-ba6c-c33ea41dc28a service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:29.664938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7aed3e-3f5b-43b9-ba6c-c33ea41dc28a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:29.674601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7aed3e-3f5b-43b9-ba6c-c33ea41dc28a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:29.676150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7aed3e-3f5b-43b9-ba6c-c33ea41dc28a service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:29.685149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1113/2224] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:29.745402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa559f8-5200-4a8c-bb59-dcc21cada2e4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:29.756827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa559f8-5200-4a8c-bb59-dcc21cada2e4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:29.762648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b884da3-83ad-403d-974c-bd9271b8aa5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:29.763738 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b884da3-83ad-403d-974c-bd9271b8aa5f service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:29.764076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b884da3-83ad-403d-974c-bd9271b8aa5f service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:29.764433 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b884da3-83ad-403d-974c-bd9271b8aa5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:29.773553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b884da3-83ad-403d-974c-bd9271b8aa5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:29.775125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b884da3-83ad-403d-974c-bd9271b8aa5f service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:29.785551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1112/2225] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:29.802313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7aed3e-3f5b-43b9-ba6c-c33ea41dc28a service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:29.900053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7aed3e-3f5b-43b9-ba6c-c33ea41dc28a service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:29.905845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:29.907102 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:29.907374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:29.907645 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] PATH_INFO: `/v3/users/d15d5cc17aff4fd885ee56e8eee296c4` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:29.917940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d15d5cc17aff4fd885ee56e8eee296c4)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:29.917940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:30.132613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd15d5cc17aff4fd885ee56e8eee296c4'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:30.142132 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd15d5cc17aff4fd885ee56e8eee296c4'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:30.142901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd15d5cc17aff4fd885ee56e8eee296c4'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:30.149015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:30.164392 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd15d5cc17aff4fd885ee56e8eee296c4'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:30.263049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd15d5cc17aff4fd885ee56e8eee296c4'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:30.278652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1114/2226] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:29 2022] DELETE /identity/v3/users/d15d5cc17aff4fd885ee56e8eee296c4 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:30.283386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fac14-5ac8-4b4f-9cf0-016cbf0f8f22 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:30.283822 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fac14-5ac8-4b4f-9cf0-016cbf0f8f22 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:30.283947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fac14-5ac8-4b4f-9cf0-016cbf0f8f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:30.303170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-311fac14-5ac8-4b4f-9cf0-016cbf0f8f22 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:30.304470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-311fac14-5ac8-4b4f-9cf0-016cbf0f8f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:30.329959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:30.365895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1113/2227] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:30.375818 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:30.377280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:30.441299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760074d-4219-40e8-9f28-8e967eda9289 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:30.444962 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09facb9-1e04-4c55-aabc-0701fccc8cb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:30.445615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09facb9-1e04-4c55-aabc-0701fccc8cb7 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:30.445894 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09facb9-1e04-4c55-aabc-0701fccc8cb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:30.446171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09facb9-1e04-4c55-aabc-0701fccc8cb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:30.456999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09facb9-1e04-4c55-aabc-0701fccc8cb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:30.458440 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09facb9-1e04-4c55-aabc-0701fccc8cb7 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:30.469596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1115/2228] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:30.925366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fac14-5ac8-4b4f-9cf0-016cbf0f8f22 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:30.932356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-311fac14-5ac8-4b4f-9cf0-016cbf0f8f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:30.955479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fac14-5ac8-4b4f-9cf0-016cbf0f8f22 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:30.960533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae371336-f175-45ad-80dd-01f08f8915a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:30.961528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae371336-f175-45ad-80dd-01f08f8915a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:30.961736 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae371336-f175-45ad-80dd-01f08f8915a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:30.962057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae371336-f175-45ad-80dd-01f08f8915a4 admin admin] PATH_INFO: `/v3/projects/39ca39bc5f714883b141c96b65d6e826` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:30.963836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae371336-f175-45ad-80dd-01f08f8915a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ca39bc5f714883b141c96b65d6e826)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:30.968300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae371336-f175-45ad-80dd-01f08f8915a4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:31.110781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ae371336-f175-45ad-80dd-01f08f8915a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ca39bc5f714883b141c96b65d6e826', 'request_id': 'req-ae371336-f175-45ad-80dd-01f08f8915a4'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:31.211813 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ae371336-f175-45ad-80dd-01f08f8915a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ca39bc5f714883b141c96b65d6e826', 'request_id': 'req-ae371336-f175-45ad-80dd-01f08f8915a4'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:31.219572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ae371336-f175-45ad-80dd-01f08f8915a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ca39bc5f714883b141c96b65d6e826', 'request_id': 'req-ae371336-f175-45ad-80dd-01f08f8915a4'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:31.233164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ae371336-f175-45ad-80dd-01f08f8915a4 admin admin] The token cache is being invalidate because project 39ca39bc5f714883b141c96b65d6e826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:31.233449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ae371336-f175-45ad-80dd-01f08f8915a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:31.234154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ae371336-f175-45ad-80dd-01f08f8915a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:31.235704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1114/2229] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:30 2022] DELETE /identity/v3/projects/39ca39bc5f714883b141c96b65d6e826 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:31.241732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518f31c-b101-4c65-ab38-7115614b22b9 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:31.242238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518f31c-b101-4c65-ab38-7115614b22b9 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:31.242238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518f31c-b101-4c65-ab38-7115614b22b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:31.263031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f518f31c-b101-4c65-ab38-7115614b22b9 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:31.264265 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f518f31c-b101-4c65-ab38-7115614b22b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:31.356611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1116/2230] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:31.364788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95b22a-7012-4ba8-a5c5-434b7d08b2d5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:31.365306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95b22a-7012-4ba8-a5c5-434b7d08b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:31.365386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95b22a-7012-4ba8-a5c5-434b7d08b2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:31.389821 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4d95b22a-7012-4ba8-a5c5-434b7d08b2d5 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:31.391202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4d95b22a-7012-4ba8-a5c5-434b7d08b2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:31.400043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1115/2231] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:31.404431 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518f31c-b101-4c65-ab38-7115614b22b9 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:31.414942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518f31c-b101-4c65-ab38-7115614b22b9 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:31.422576 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37238d18-8de5-4e3d-9619-8823f8b21aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:31.423534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37238d18-8de5-4e3d-9619-8823f8b21aef admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:31.423820 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37238d18-8de5-4e3d-9619-8823f8b21aef admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:31.424097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37238d18-8de5-4e3d-9619-8823f8b21aef admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:31.424839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37238d18-8de5-4e3d-9619-8823f8b21aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:01:31.425140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37238d18-8de5-4e3d-9619-8823f8b21aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:31.426771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37238d18-8de5-4e3d-9619-8823f8b21aef admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:31.444600 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1117/2232] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:01:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:01:31.453122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95b22a-7012-4ba8-a5c5-434b7d08b2d5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:31.463088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95b22a-7012-4ba8-a5c5-434b7d08b2d5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:31.468312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68cd08-8fb9-4e09-834c-5bd6f4115022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:31.468986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68cd08-8fb9-4e09-834c-5bd6f4115022 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:31.469271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68cd08-8fb9-4e09-834c-5bd6f4115022 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:31.469547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68cd08-8fb9-4e09-834c-5bd6f4115022 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:31.470253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68cd08-8fb9-4e09-834c-5bd6f4115022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:31.472232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68cd08-8fb9-4e09-834c-5bd6f4115022 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:31.563630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1116/2233] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:01:31 2022] POST /identity/v3/projects => generated 368 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:01:31.567831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37238d18-8de5-4e3d-9619-8823f8b21aef admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:31.578802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37238d18-8de5-4e3d-9619-8823f8b21aef admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:31.644218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbdf0e-75c5-4523-a6df-fef1d39bfc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:31.644897 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbdf0e-75c5-4523-a6df-fef1d39bfc0c admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:31.645174 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbdf0e-75c5-4523-a6df-fef1d39bfc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:31.645448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbdf0e-75c5-4523-a6df-fef1d39bfc0c admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:31.646149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbdf0e-75c5-4523-a6df-fef1d39bfc0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:31.648119 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbdf0e-75c5-4523-a6df-fef1d39bfc0c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:31.717680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68cd08-8fb9-4e09-834c-5bd6f4115022 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:31.720307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ac68cd08-8fb9-4e09-834c-5bd6f4115022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:31.735008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68cd08-8fb9-4e09-834c-5bd6f4115022 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:31.740272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bd1e1-0cf5-477b-b835-125eadb08e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:31.741226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bd1e1-0cf5-477b-b835-125eadb08e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:31.741511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bd1e1-0cf5-477b-b835-125eadb08e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:31.741797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bd1e1-0cf5-477b-b835-125eadb08e49 admin admin] PATH_INFO: `/v3/users/556aab8b6cb04d06bbf49250a456aaf0` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:31.745893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0bd1e1-0cf5-477b-b835-125eadb08e49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=556aab8b6cb04d06bbf49250a456aaf0)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:31.745893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0bd1e1-0cf5-477b-b835-125eadb08e49 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:31.767934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:31.773003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1118/2234] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:01:31 2022] POST /identity/v3/users => generated 285 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:31.777513 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbdf0e-75c5-4523-a6df-fef1d39bfc0c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:31.787899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbdf0e-75c5-4523-a6df-fef1d39bfc0c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:31.793775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42252187-f863-4a43-969f-9272238b73d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:31.794471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42252187-f863-4a43-969f-9272238b73d4 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:31.799028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42252187-f863-4a43-969f-9272238b73d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:31.799028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42252187-f863-4a43-969f-9272238b73d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:31.799028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42252187-f863-4a43-969f-9272238b73d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:31.799028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42252187-f863-4a43-969f-9272238b73d4 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:31.810137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1119/2235] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:31 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:01:31.814707 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42252187-f863-4a43-969f-9272238b73d4 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:31.828287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42252187-f863-4a43-969f-9272238b73d4 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:31.836665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d534af-177e-4590-8db4-ebba5f5128ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:31.836665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d534af-177e-4590-8db4-ebba5f5128ae admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:31.836665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d534af-177e-4590-8db4-ebba5f5128ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:31.836665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d534af-177e-4590-8db4-ebba5f5128ae admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:31.836989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d534af-177e-4590-8db4-ebba5f5128ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:31.836989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d534af-177e-4590-8db4-ebba5f5128ae admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:31.848761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1120/2236] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:31 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:31.852821 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d534af-177e-4590-8db4-ebba5f5128ae admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:31.864169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d534af-177e-4590-8db4-ebba5f5128ae admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:31.869444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216c2f9-17d5-4875-b5ff-74568ae6b912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:31.871032 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216c2f9-17d5-4875-b5ff-74568ae6b912 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:31.871331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216c2f9-17d5-4875-b5ff-74568ae6b912 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:31.871611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216c2f9-17d5-4875-b5ff-74568ae6b912 admin admin] PATH_INFO: `/v3/projects/559181a2d6954b9c9b3fc6fc228c0d1f/users/63768186635d4e4bbbb3efdd546cef0c/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:31.888449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216c2f9-17d5-4875-b5ff-74568ae6b912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=559181a2d6954b9c9b3fc6fc228c0d1f, user_id=63768186635d4e4bbbb3efdd546cef0c, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:31.891317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216c2f9-17d5-4875-b5ff-74568ae6b912 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:31.952388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0e0bd1e1-0cf5-477b-b835-125eadb08e49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '556aab8b6cb04d06bbf49250a456aaf0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:31.959859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0e0bd1e1-0cf5-477b-b835-125eadb08e49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '556aab8b6cb04d06bbf49250a456aaf0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:31.960524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0e0bd1e1-0cf5-477b-b835-125eadb08e49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '556aab8b6cb04d06bbf49250a456aaf0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:31.976786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0e0bd1e1-0cf5-477b-b835-125eadb08e49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '556aab8b6cb04d06bbf49250a456aaf0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:31.981335 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:31.989163 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1121/2237] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:31 2022] PUT /identity/v3/projects/559181a2d6954b9c9b3fc6fc228c0d1f/users/63768186635d4e4bbbb3efdd546cef0c/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:01:31.999874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c28e7-ffda-488b-8771-390dd4e35cf0 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:31.999874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c28e7-ffda-488b-8771-390dd4e35cf0 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:32.000096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c28e7-ffda-488b-8771-390dd4e35cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:32.034573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-663c28e7-ffda-488b-8771-390dd4e35cf0 None None] MFA Rules not processed for user `63768186635d4e4bbbb3efdd546cef0c`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:32.035869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-663c28e7-ffda-488b-8771-390dd4e35cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:32.037418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0e0bd1e1-0cf5-477b-b835-125eadb08e49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '556aab8b6cb04d06bbf49250a456aaf0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:32.046783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1117/2238] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:31 2022] DELETE /identity/v3/users/556aab8b6cb04d06bbf49250a456aaf0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:01:32.051184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abdce2-ead4-44be-ba69-09750819316b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:32.051476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abdce2-ead4-44be-ba69-09750819316b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:32.051754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abdce2-ead4-44be-ba69-09750819316b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:32.069693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-79abdce2-ead4-44be-ba69-09750819316b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:32.070946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-79abdce2-ead4-44be-ba69-09750819316b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:32.115007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1122/2239] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:01:32.127836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c28e7-ffda-488b-8771-390dd4e35cf0 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:32.129230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-663c28e7-ffda-488b-8771-390dd4e35cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:32.141389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1118/2240] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:01:32.150468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abdce2-ead4-44be-ba69-09750819316b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:32.156247 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-79abdce2-ead4-44be-ba69-09750819316b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:32.198622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c28e7-ffda-488b-8771-390dd4e35cf0 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:32.217685 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abdce2-ead4-44be-ba69-09750819316b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:32.222162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6782631-020f-4a35-8c82-a63a66d7f47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:32.222834 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6782631-020f-4a35-8c82-a63a66d7f47a service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:32.223122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6782631-020f-4a35-8c82-a63a66d7f47a service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:32.223394 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6782631-020f-4a35-8c82-a63a66d7f47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:32.233333 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6782631-020f-4a35-8c82-a63a66d7f47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:32.234769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6782631-020f-4a35-8c82-a63a66d7f47a service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:32.245882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1119/2241] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:32.255238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4d080-80f0-4135-842d-9b6062f7df40 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '683c54f489f440b9842cb94980214df1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:32.255238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4d080-80f0-4135-842d-9b6062f7df40 service glance] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:32.255238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4d080-80f0-4135-842d-9b6062f7df40 service glance] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:32.255238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4d080-80f0-4135-842d-9b6062f7df40 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:32.267081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:32.319050 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4d080-80f0-4135-842d-9b6062f7df40 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:32.320499 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4d080-80f0-4135-842d-9b6062f7df40 service glance] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:32.331705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1123/2242] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:01:32.474412 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6782631-020f-4a35-8c82-a63a66d7f47a service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:32.476058 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a6782631-020f-4a35-8c82-a63a66d7f47a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:32.498178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6782631-020f-4a35-8c82-a63a66d7f47a service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:32.512351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb32d70-d5b3-4852-995a-78e1035b85c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:32.513800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb32d70-d5b3-4852-995a-78e1035b85c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:32.514268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb32d70-d5b3-4852-995a-78e1035b85c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:32.514729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb32d70-d5b3-4852-995a-78e1035b85c2 admin admin] PATH_INFO: `/v3/projects/afe657943fa44b88baffcf9fd273b4e9` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:32.516867 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb32d70-d5b3-4852-995a-78e1035b85c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe657943fa44b88baffcf9fd273b4e9)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:32.519135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb32d70-d5b3-4852-995a-78e1035b85c2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:32.605996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c228f6-84f7-4f0f-a812-38053237008a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:32.606312 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c228f6-84f7-4f0f-a812-38053237008a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:32.612077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c228f6-84f7-4f0f-a812-38053237008a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:32.627065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-10c228f6-84f7-4f0f-a812-38053237008a None None] MFA Rules not processed for user `9dc1b321e70f42e2be177e161486f997`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:32.628291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-10c228f6-84f7-4f0f-a812-38053237008a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:32.677045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ceb32d70-d5b3-4852-995a-78e1035b85c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe657943fa44b88baffcf9fd273b4e9', 'request_id': 'req-ceb32d70-d5b3-4852-995a-78e1035b85c2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:32.709685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1124/2243] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:01:32.763824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ceb32d70-d5b3-4852-995a-78e1035b85c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe657943fa44b88baffcf9fd273b4e9', 'request_id': 'req-ceb32d70-d5b3-4852-995a-78e1035b85c2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:32.773932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ceb32d70-d5b3-4852-995a-78e1035b85c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe657943fa44b88baffcf9fd273b4e9', 'request_id': 'req-ceb32d70-d5b3-4852-995a-78e1035b85c2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:32.787704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ceb32d70-d5b3-4852-995a-78e1035b85c2 admin admin] The token cache is being invalidate because project afe657943fa44b88baffcf9fd273b4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:32.788236 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ceb32d70-d5b3-4852-995a-78e1035b85c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:32.789058 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ceb32d70-d5b3-4852-995a-78e1035b85c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:32.791291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1120/2244] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:32 2022] DELETE /identity/v3/projects/afe657943fa44b88baffcf9fd273b4e9 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:32.796780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7fb35-7042-4d97-bd6f-e7a11d7d1a4f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:32.797071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7fb35-7042-4d97-bd6f-e7a11d7d1a4f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:32.797344 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7fb35-7042-4d97-bd6f-e7a11d7d1a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:32.818247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b5a7fb35-7042-4d97-bd6f-e7a11d7d1a4f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:32.819553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b5a7fb35-7042-4d97-bd6f-e7a11d7d1a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:32.894400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1125/2245] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:32.902521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52015b2f-007a-490b-bb70-a86d1708b596 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:32.902807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52015b2f-007a-490b-bb70-a86d1708b596 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:32.903142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52015b2f-007a-490b-bb70-a86d1708b596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:32.942652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-52015b2f-007a-490b-bb70-a86d1708b596 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:32.943942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-52015b2f-007a-490b-bb70-a86d1708b596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:32.955172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1121/2246] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:01:32.959537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7fb35-7042-4d97-bd6f-e7a11d7d1a4f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:32.972468 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7fb35-7042-4d97-bd6f-e7a11d7d1a4f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:32.977689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05d768-562c-4dbd-ae42-55fd0c70d37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:32.978564 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05d768-562c-4dbd-ae42-55fd0c70d37e admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:32.979081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05d768-562c-4dbd-ae42-55fd0c70d37e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:32.983881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05d768-562c-4dbd-ae42-55fd0c70d37e admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:32.984941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05d768-562c-4dbd-ae42-55fd0c70d37e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:01:32.985356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05d768-562c-4dbd-ae42-55fd0c70d37e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:32.987116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05d768-562c-4dbd-ae42-55fd0c70d37e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:33.004766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52015b2f-007a-490b-bb70-a86d1708b596 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:33.012349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1126/2247] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:01:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:01:33.018760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52015b2f-007a-490b-bb70-a86d1708b596 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:33.024749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c849b95-1278-4157-b4c6-860457ffc1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:33.025568 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c849b95-1278-4157-b4c6-860457ffc1bd admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:33.025804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c849b95-1278-4157-b4c6-860457ffc1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:33.026176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c849b95-1278-4157-b4c6-860457ffc1bd admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:33.028131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c849b95-1278-4157-b4c6-860457ffc1bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:33.029953 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c849b95-1278-4157-b4c6-860457ffc1bd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:33.047144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:33.123391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1122/2248] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:01:33 2022] POST /identity/v3/projects => generated 384 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:01:33.127773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05d768-562c-4dbd-ae42-55fd0c70d37e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:33.151818 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05d768-562c-4dbd-ae42-55fd0c70d37e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:33.211042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:33.225444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82145e8-b4b2-428a-9f50-9d21dfa0d6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:33.226254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82145e8-b4b2-428a-9f50-9d21dfa0d6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:33.226531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82145e8-b4b2-428a-9f50-9d21dfa0d6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:33.226969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82145e8-b4b2-428a-9f50-9d21dfa0d6d4 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:33.228494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82145e8-b4b2-428a-9f50-9d21dfa0d6d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:33.230840 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82145e8-b4b2-428a-9f50-9d21dfa0d6d4 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:33.327513 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1127/2249] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:01:33 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:01:33.331713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c849b95-1278-4157-b4c6-860457ffc1bd admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:33.343938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c849b95-1278-4157-b4c6-860457ffc1bd admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:33.348690 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402e21a-5a15-4a68-b06c-f3ac0e581d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:33.349397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402e21a-5a15-4a68-b06c-f3ac0e581d45 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:33.349677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402e21a-5a15-4a68-b06c-f3ac0e581d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:33.349953 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402e21a-5a15-4a68-b06c-f3ac0e581d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:33.350710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402e21a-5a15-4a68-b06c-f3ac0e581d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:33.352457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402e21a-5a15-4a68-b06c-f3ac0e581d45 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:33.363900 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1123/2250] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:33 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:33.368183 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82145e8-b4b2-428a-9f50-9d21dfa0d6d4 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:33.380830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82145e8-b4b2-428a-9f50-9d21dfa0d6d4 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:33.388197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882450b-843c-4ec4-90af-ed331bdd221a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:33.388895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882450b-843c-4ec4-90af-ed331bdd221a admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:33.389179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882450b-843c-4ec4-90af-ed331bdd221a admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:33.389463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882450b-843c-4ec4-90af-ed331bdd221a admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:33.390327 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882450b-843c-4ec4-90af-ed331bdd221a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:33.392435 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882450b-843c-4ec4-90af-ed331bdd221a admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:33.408180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1128/2251] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:33 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:33.416270 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402e21a-5a15-4a68-b06c-f3ac0e581d45 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:33.426963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402e21a-5a15-4a68-b06c-f3ac0e581d45 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:33.433069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e71f5-1141-4b37-a16f-02fc823fd5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:33.434308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e71f5-1141-4b37-a16f-02fc823fd5f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:33.434605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e71f5-1141-4b37-a16f-02fc823fd5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:33.434915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e71f5-1141-4b37-a16f-02fc823fd5f2 admin admin] PATH_INFO: `/v3/projects/ed709461c5a0499abc1a9cc33a8f916d/users/99f2374b551741f4bc602debd1df336b/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:33.460855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e71f5-1141-4b37-a16f-02fc823fd5f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed709461c5a0499abc1a9cc33a8f916d, user_id=99f2374b551741f4bc602debd1df336b, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:33.464542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e71f5-1141-4b37-a16f-02fc823fd5f2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:33.529577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1124/2252] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:33 2022] PUT /identity/v3/projects/ed709461c5a0499abc1a9cc33a8f916d/users/99f2374b551741f4bc602debd1df336b/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:33.538227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5067169-c605-4e57-a305-acc4bb47af0e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:33.539008 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5067169-c605-4e57-a305-acc4bb47af0e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:33.539531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5067169-c605-4e57-a305-acc4bb47af0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:33.565538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a5067169-c605-4e57-a305-acc4bb47af0e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:33.570176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a5067169-c605-4e57-a305-acc4bb47af0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:33.654596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1129/2253] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:33.661772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e71f5-1141-4b37-a16f-02fc823fd5f2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:33.663647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-510e71f5-1141-4b37-a16f-02fc823fd5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:33.731901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e71f5-1141-4b37-a16f-02fc823fd5f2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:33.735866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f2c3d-58b2-4ba3-bf63-ce1d4749b7a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:33.736551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f2c3d-58b2-4ba3-bf63-ce1d4749b7a8 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:33.736825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f2c3d-58b2-4ba3-bf63-ce1d4749b7a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:33.737096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f2c3d-58b2-4ba3-bf63-ce1d4749b7a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:33.747564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f2c3d-58b2-4ba3-bf63-ce1d4749b7a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:33.749167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f2c3d-58b2-4ba3-bf63-ce1d4749b7a8 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:33.760278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1125/2254] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:35.199557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbae298-c3b8-4289-861e-8231b7089691 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:35.199557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbae298-c3b8-4289-861e-8231b7089691 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:35.200209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbae298-c3b8-4289-861e-8231b7089691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:35.228725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-6dbae298-c3b8-4289-861e-8231b7089691 None None] MFA Rules not processed for user `99f2374b551741f4bc602debd1df336b`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:35.230719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6dbae298-c3b8-4289-861e-8231b7089691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:35.317853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1130/2255] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:35.325078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f2c3d-58b2-4ba3-bf63-ce1d4749b7a8 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:35.326750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8b9f2c3d-58b2-4ba3-bf63-ce1d4749b7a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:35.387895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f2c3d-58b2-4ba3-bf63-ce1d4749b7a8 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:35.391705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16992aca-8a7e-4547-9f71-6008749903bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:35.392462 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16992aca-8a7e-4547-9f71-6008749903bb service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:35.392813 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16992aca-8a7e-4547-9f71-6008749903bb service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:35.393176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16992aca-8a7e-4547-9f71-6008749903bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:35.402090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16992aca-8a7e-4547-9f71-6008749903bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:35.403567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16992aca-8a7e-4547-9f71-6008749903bb service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:35.412739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1126/2256] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:41.336206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0e571-ecd0-4d65-b2ab-4e7739894744 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:41.336726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0e571-ecd0-4d65-b2ab-4e7739894744 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:41.336816 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0e571-ecd0-4d65-b2ab-4e7739894744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:41.364196 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d4f0e571-ecd0-4d65-b2ab-4e7739894744 None None] MFA Rules not processed for user `99f2374b551741f4bc602debd1df336b`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:41.365460 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d4f0e571-ecd0-4d65-b2ab-4e7739894744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:41.376291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1131/2257] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:44.075248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16992aca-8a7e-4547-9f71-6008749903bb service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:44.076931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-16992aca-8a7e-4547-9f71-6008749903bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:44.097928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16992aca-8a7e-4547-9f71-6008749903bb service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:44.105265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:44.107286 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:44.107835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:44.108310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] PATH_INFO: `/v3/users/a176a96e7e4e483ea1aeeb9e6ee44239` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:44.109776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a176a96e7e4e483ea1aeeb9e6ee44239)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:44.128930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:44.307465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a176a96e7e4e483ea1aeeb9e6ee44239'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:44.315364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a176a96e7e4e483ea1aeeb9e6ee44239'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:44.319507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a176a96e7e4e483ea1aeeb9e6ee44239'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:44.319507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:44.331224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a176a96e7e4e483ea1aeeb9e6ee44239'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:44.420081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a176a96e7e4e483ea1aeeb9e6ee44239'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:44.427633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1127/2258] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:44 2022] DELETE /identity/v3/users/a176a96e7e4e483ea1aeeb9e6ee44239 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:44.434493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d89ae-174a-499c-af23-70c86f7742e3 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:44.434493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d89ae-174a-499c-af23-70c86f7742e3 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:44.434493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d89ae-174a-499c-af23-70c86f7742e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:44.452511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-af8d89ae-174a-499c-af23-70c86f7742e3 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:44.453874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-af8d89ae-174a-499c-af23-70c86f7742e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:44.507171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:44.523557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1132/2259] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:44.534064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:44.536237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:44.597903 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aec80f-a997-4cbd-84d7-b598017b688d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:44.602122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb90ab6-70c6-4d02-9cc2-515886724029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:44.602787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb90ab6-70c6-4d02-9cc2-515886724029 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:44.603103 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb90ab6-70c6-4d02-9cc2-515886724029 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:44.603387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb90ab6-70c6-4d02-9cc2-515886724029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:44.613872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb90ab6-70c6-4d02-9cc2-515886724029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:44.615389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb90ab6-70c6-4d02-9cc2-515886724029 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:44.625093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1128/2260] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:01:45.325502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d89ae-174a-499c-af23-70c86f7742e3 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:45.325502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-af8d89ae-174a-499c-af23-70c86f7742e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:45.351343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d89ae-174a-499c-af23-70c86f7742e3 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:45.369106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:45.370343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:45.371091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:45.371091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] PATH_INFO: `/v3/projects/8b062a27c4c244ef9107cd2027269009` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:45.373091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b062a27c4c244ef9107cd2027269009)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:45.375402 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:45.534152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b062a27c4c244ef9107cd2027269009', 'request_id': 'req-011653e0-0b2e-4dfc-b4fc-df896dc44036'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:45.663434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b062a27c4c244ef9107cd2027269009', 'request_id': 'req-011653e0-0b2e-4dfc-b4fc-df896dc44036'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:45.671670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b062a27c4c244ef9107cd2027269009', 'request_id': 'req-011653e0-0b2e-4dfc-b4fc-df896dc44036'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:45.685205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] The token cache is being invalidate because project 8b062a27c4c244ef9107cd2027269009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:45.685497 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:45.686195 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:45.687810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1133/2261] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:45 2022] DELETE /identity/v3/projects/8b062a27c4c244ef9107cd2027269009 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:45.691666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb90ab6-70c6-4d02-9cc2-515886724029 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:45.704217 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb90ab6-70c6-4d02-9cc2-515886724029 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:45.796633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed0702-1db3-425c-ad71-dd82a8516abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:45.797529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed0702-1db3-425c-ad71-dd82a8516abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:45.797806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed0702-1db3-425c-ad71-dd82a8516abd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:45.798078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed0702-1db3-425c-ad71-dd82a8516abd admin admin] PATH_INFO: `/v3/users/9c54c05453294e7093430c6055cbefaf` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:45.799345 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed0702-1db3-425c-ad71-dd82a8516abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c54c05453294e7093430c6055cbefaf)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:45.801316 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed0702-1db3-425c-ad71-dd82a8516abd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:46.057520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-69ed0702-1db3-425c-ad71-dd82a8516abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c54c05453294e7093430c6055cbefaf'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:46.071628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-69ed0702-1db3-425c-ad71-dd82a8516abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c54c05453294e7093430c6055cbefaf'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:46.073287 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-69ed0702-1db3-425c-ad71-dd82a8516abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c54c05453294e7093430c6055cbefaf'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:46.089041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-69ed0702-1db3-425c-ad71-dd82a8516abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c54c05453294e7093430c6055cbefaf'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:46.149468 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-69ed0702-1db3-425c-ad71-dd82a8516abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c54c05453294e7093430c6055cbefaf'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:46.151654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:46.161102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1129/2262] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:01:45 2022] DELETE /identity/v3/users/9c54c05453294e7093430c6055cbefaf => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:46.679185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:46.680707 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:46.700047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011653e0-0b2e-4dfc-b4fc-df896dc44036 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:46.762422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23581a74-1795-46a2-a75b-a0eb37aa7d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:46.764058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23581a74-1795-46a2-a75b-a0eb37aa7d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:46.766237 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23581a74-1795-46a2-a75b-a0eb37aa7d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:46.766717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23581a74-1795-46a2-a75b-a0eb37aa7d2f admin admin] PATH_INFO: `/v3/projects/8e7feaab64dc42cc85fff5f23b37d442` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:46.768412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23581a74-1795-46a2-a75b-a0eb37aa7d2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e7feaab64dc42cc85fff5f23b37d442)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:46.770397 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23581a74-1795-46a2-a75b-a0eb37aa7d2f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:46.883361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-23581a74-1795-46a2-a75b-a0eb37aa7d2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e7feaab64dc42cc85fff5f23b37d442', 'request_id': 'req-23581a74-1795-46a2-a75b-a0eb37aa7d2f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:46.946858 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-23581a74-1795-46a2-a75b-a0eb37aa7d2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e7feaab64dc42cc85fff5f23b37d442', 'request_id': 'req-23581a74-1795-46a2-a75b-a0eb37aa7d2f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:46.957055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-23581a74-1795-46a2-a75b-a0eb37aa7d2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e7feaab64dc42cc85fff5f23b37d442', 'request_id': 'req-23581a74-1795-46a2-a75b-a0eb37aa7d2f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:46.969180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-23581a74-1795-46a2-a75b-a0eb37aa7d2f admin admin] The token cache is being invalidate because project 8e7feaab64dc42cc85fff5f23b37d442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:01:46.969689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-23581a74-1795-46a2-a75b-a0eb37aa7d2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:46.970350 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-23581a74-1795-46a2-a75b-a0eb37aa7d2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:01:46.972197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1134/2263] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:01:46 2022] DELETE /identity/v3/projects/8e7feaab64dc42cc85fff5f23b37d442 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:01:46.977398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0408c1f-c8c1-4bb6-9eec-81ed7a2fac71 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:46.981322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0408c1f-c8c1-4bb6-9eec-81ed7a2fac71 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:46.981322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0408c1f-c8c1-4bb6-9eec-81ed7a2fac71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:47.003603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c0408c1f-c8c1-4bb6-9eec-81ed7a2fac71 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:47.005038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c0408c1f-c8c1-4bb6-9eec-81ed7a2fac71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:47.070171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1130/2264] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:47.078738 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32968b2-f8b9-4b18-bc37-c40db2bf4733 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:47.079047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32968b2-f8b9-4b18-bc37-c40db2bf4733 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:47.079339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32968b2-f8b9-4b18-bc37-c40db2bf4733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:47.085939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:47.099095 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d32968b2-f8b9-4b18-bc37-c40db2bf4733 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:47.100332 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d32968b2-f8b9-4b18-bc37-c40db2bf4733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:47.108705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1135/2265] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:47.113032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0408c1f-c8c1-4bb6-9eec-81ed7a2fac71 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:47.125137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0408c1f-c8c1-4bb6-9eec-81ed7a2fac71 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:47.130059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10f345-4a61-4032-9775-144f0db0ae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:47.130787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10f345-4a61-4032-9775-144f0db0ae39 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:47.131162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10f345-4a61-4032-9775-144f0db0ae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:47.131439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10f345-4a61-4032-9775-144f0db0ae39 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:47.132169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10f345-4a61-4032-9775-144f0db0ae39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:01:47.132472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10f345-4a61-4032-9775-144f0db0ae39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:47.134248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10f345-4a61-4032-9775-144f0db0ae39 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:47.150727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1131/2266] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:01:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:01:47.152031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32968b2-f8b9-4b18-bc37-c40db2bf4733 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:47.161545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32968b2-f8b9-4b18-bc37-c40db2bf4733 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:47.166747 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0b47b-14bd-44ea-bfd0-d0c95bb9efcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:47.167470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0b47b-14bd-44ea-bfd0-d0c95bb9efcd admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:47.167830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0b47b-14bd-44ea-bfd0-d0c95bb9efcd admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:47.168096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0b47b-14bd-44ea-bfd0-d0c95bb9efcd admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:47.168795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0b47b-14bd-44ea-bfd0-d0c95bb9efcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:47.170866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0b47b-14bd-44ea-bfd0-d0c95bb9efcd admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:47.279439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1136/2267] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:01:47 2022] POST /identity/v3/projects => generated 378 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:47.283764 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10f345-4a61-4032-9775-144f0db0ae39 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:47.295300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10f345-4a61-4032-9775-144f0db0ae39 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:47.363298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71231e63-16a3-4489-9a63-0b31a2175f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:47.363983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71231e63-16a3-4489-9a63-0b31a2175f58 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:47.364267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71231e63-16a3-4489-9a63-0b31a2175f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:47.364545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71231e63-16a3-4489-9a63-0b31a2175f58 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:47.365883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71231e63-16a3-4489-9a63-0b31a2175f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:47.367858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71231e63-16a3-4489-9a63-0b31a2175f58 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:47.533833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1132/2268] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:01:47 2022] POST /identity/v3/users => generated 290 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:01:47.538076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0b47b-14bd-44ea-bfd0-d0c95bb9efcd admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:47.552281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0b47b-14bd-44ea-bfd0-d0c95bb9efcd admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:47.557225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18749e2-9de2-4fbf-be26-652015b4cbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:47.557959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18749e2-9de2-4fbf-be26-652015b4cbff admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:47.558220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18749e2-9de2-4fbf-be26-652015b4cbff admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:47.559107 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18749e2-9de2-4fbf-be26-652015b4cbff admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:47.560179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18749e2-9de2-4fbf-be26-652015b4cbff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:47.561448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18749e2-9de2-4fbf-be26-652015b4cbff admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:47.572449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1137/2269] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:47 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:47.576993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71231e63-16a3-4489-9a63-0b31a2175f58 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:47.586974 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71231e63-16a3-4489-9a63-0b31a2175f58 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:47.596200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061ef12-dcae-438f-9751-bf59a00a7bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:47.596888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061ef12-dcae-438f-9751-bf59a00a7bce admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:47.597168 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061ef12-dcae-438f-9751-bf59a00a7bce admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:47.597443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061ef12-dcae-438f-9751-bf59a00a7bce admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:47.598173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061ef12-dcae-438f-9751-bf59a00a7bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:47.599797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061ef12-dcae-438f-9751-bf59a00a7bce admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:47.611507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1133/2270] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:47 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:47.616096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18749e2-9de2-4fbf-be26-652015b4cbff admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:47.626303 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18749e2-9de2-4fbf-be26-652015b4cbff admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:47.634688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc99db5-efe6-4f48-900b-d797550a9705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:47.635675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc99db5-efe6-4f48-900b-d797550a9705 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:47.635958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc99db5-efe6-4f48-900b-d797550a9705 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:47.636231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc99db5-efe6-4f48-900b-d797550a9705 admin admin] PATH_INFO: `/v3/projects/b155fe54367448d08baceb74fa8c1206/users/3d27063d8dfd4c57823a7e40aa254906/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:47.653706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc99db5-efe6-4f48-900b-d797550a9705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b155fe54367448d08baceb74fa8c1206, user_id=3d27063d8dfd4c57823a7e40aa254906, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:47.656390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc99db5-efe6-4f48-900b-d797550a9705 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:47.734204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1138/2271] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:47 2022] PUT /identity/v3/projects/b155fe54367448d08baceb74fa8c1206/users/3d27063d8dfd4c57823a7e40aa254906/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:47.745203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa781256-5869-42b0-bcbe-d220df5381fd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:47.750866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa781256-5869-42b0-bcbe-d220df5381fd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:47.750866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa781256-5869-42b0-bcbe-d220df5381fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:47.782015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-fa781256-5869-42b0-bcbe-d220df5381fd None None] MFA Rules not processed for user `3d27063d8dfd4c57823a7e40aa254906`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:47.783550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fa781256-5869-42b0-bcbe-d220df5381fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:47.842921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:47.853177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1134/2272] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:47.859321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc99db5-efe6-4f48-900b-d797550a9705 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:47.941539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc99db5-efe6-4f48-900b-d797550a9705 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:47.953400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb37b1-e0f3-4516-9a3f-81bedcda0684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:47.953400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb37b1-e0f3-4516-9a3f-81bedcda0684 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:47.953400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb37b1-e0f3-4516-9a3f-81bedcda0684 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:47.953400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb37b1-e0f3-4516-9a3f-81bedcda0684 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:47.954040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb37b1-e0f3-4516-9a3f-81bedcda0684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:47.954040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb37b1-e0f3-4516-9a3f-81bedcda0684 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:48.065920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1139/2273] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:01:47 2022] POST /identity/v3/projects => generated 378 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:48.070024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa781256-5869-42b0-bcbe-d220df5381fd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:48.081666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa781256-5869-42b0-bcbe-d220df5381fd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:48.154975 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2a08b-1f3e-4309-940d-011dccebae5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:48.155661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2a08b-1f3e-4309-940d-011dccebae5f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:48.159890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2a08b-1f3e-4309-940d-011dccebae5f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:48.159890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2a08b-1f3e-4309-940d-011dccebae5f admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:48.159890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2a08b-1f3e-4309-940d-011dccebae5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:48.159890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2a08b-1f3e-4309-940d-011dccebae5f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:48.256043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1135/2274] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:01:48 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:01:48.259904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb37b1-e0f3-4516-9a3f-81bedcda0684 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:48.270747 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb37b1-e0f3-4516-9a3f-81bedcda0684 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:48.275256 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeabea3-84e1-4481-9c6a-316cfef2f99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:48.275915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeabea3-84e1-4481-9c6a-316cfef2f99f admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:48.276195 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeabea3-84e1-4481-9c6a-316cfef2f99f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:48.276532 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeabea3-84e1-4481-9c6a-316cfef2f99f admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:48.277881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeabea3-84e1-4481-9c6a-316cfef2f99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:48.279680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeabea3-84e1-4481-9c6a-316cfef2f99f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:48.292541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1140/2275] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:48 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:48.297385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2a08b-1f3e-4309-940d-011dccebae5f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:48.309906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2a08b-1f3e-4309-940d-011dccebae5f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:48.320838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedc998-6ec0-4b01-8c7c-53b5ce42cf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:48.321959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedc998-6ec0-4b01-8c7c-53b5ce42cf44 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:48.322239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedc998-6ec0-4b01-8c7c-53b5ce42cf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:48.322512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedc998-6ec0-4b01-8c7c-53b5ce42cf44 admin admin] PATH_INFO: `/v3/projects/1428cbfda9cb4b7ca83b5ffbf665909b/users/2abb242e351141fbb3e0e4117230ed0f/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:48.342196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedc998-6ec0-4b01-8c7c-53b5ce42cf44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1428cbfda9cb4b7ca83b5ffbf665909b, user_id=2abb242e351141fbb3e0e4117230ed0f, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:48.346901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedc998-6ec0-4b01-8c7c-53b5ce42cf44 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:48.452797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1136/2276] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:48 2022] PUT /identity/v3/projects/1428cbfda9cb4b7ca83b5ffbf665909b/users/2abb242e351141fbb3e0e4117230ed0f/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:48.462215 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeabea3-84e1-4481-9c6a-316cfef2f99f admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:48.475377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeabea3-84e1-4481-9c6a-316cfef2f99f admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:48.544512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff991fb-ad19-449c-8f50-d4720c243503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:48.545204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff991fb-ad19-449c-8f50-d4720c243503 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:48.545483 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff991fb-ad19-449c-8f50-d4720c243503 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:48.545756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff991fb-ad19-449c-8f50-d4720c243503 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:48.546487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff991fb-ad19-449c-8f50-d4720c243503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:48.548142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff991fb-ad19-449c-8f50-d4720c243503 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:48.559642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1141/2277] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:01:48 2022] GET /identity/v3/roles => generated 1427 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:01:48.563747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedc998-6ec0-4b01-8c7c-53b5ce42cf44 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:48.575448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedc998-6ec0-4b01-8c7c-53b5ce42cf44 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:48.581248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445396ca-b77b-490d-aaec-98e18d5887bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:48.582473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445396ca-b77b-490d-aaec-98e18d5887bb admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:48.582966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445396ca-b77b-490d-aaec-98e18d5887bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:48.583920 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445396ca-b77b-490d-aaec-98e18d5887bb admin admin] PATH_INFO: `/v3/projects/1428cbfda9cb4b7ca83b5ffbf665909b/users/2abb242e351141fbb3e0e4117230ed0f/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:48.587009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445396ca-b77b-490d-aaec-98e18d5887bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1428cbfda9cb4b7ca83b5ffbf665909b, user_id=2abb242e351141fbb3e0e4117230ed0f, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:48.590665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445396ca-b77b-490d-aaec-98e18d5887bb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:48.603190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1137/2278] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:01:48 2022] PUT /identity/v3/projects/1428cbfda9cb4b7ca83b5ffbf665909b/users/2abb242e351141fbb3e0e4117230ed0f/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:01:48.614695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f60fc7-04ca-46d9-8c77-3310e39f13e0 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:48.614999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f60fc7-04ca-46d9-8c77-3310e39f13e0 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:48.615284 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f60fc7-04ca-46d9-8c77-3310e39f13e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:48.635930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:48.646473 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f0f60fc7-04ca-46d9-8c77-3310e39f13e0 None None] MFA Rules not processed for user `2abb242e351141fbb3e0e4117230ed0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:48.647743 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f0f60fc7-04ca-46d9-8c77-3310e39f13e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:48.719258 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1142/2279] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:01:48.736609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445396ca-b77b-490d-aaec-98e18d5887bb admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:48.739555 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-445396ca-b77b-490d-aaec-98e18d5887bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:48.821449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445396ca-b77b-490d-aaec-98e18d5887bb admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:48.826590 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b4745-fa34-4d21-93f8-b001edba3ad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:48.827285 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b4745-fa34-4d21-93f8-b001edba3ad4 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:48.830750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b4745-fa34-4d21-93f8-b001edba3ad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:48.830750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b4745-fa34-4d21-93f8-b001edba3ad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:48.891854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b4745-fa34-4d21-93f8-b001edba3ad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:48.893353 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b4745-fa34-4d21-93f8-b001edba3ad4 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:48.898545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:48.906230 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1138/2280] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:01:50.780510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb5062-88fa-42a2-a7a6-c2bf077a4dad None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:50.781161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb5062-88fa-42a2-a7a6-c2bf077a4dad None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:50.781161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb5062-88fa-42a2-a7a6-c2bf077a4dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:50.801550 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d4cb5062-88fa-42a2-a7a6-c2bf077a4dad None None] MFA Rules not processed for user `dbee2da50ee6413db771071900703e29`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:01:50.802848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d4cb5062-88fa-42a2-a7a6-c2bf077a4dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:01:50.860998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1143/2281] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:01:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:01:59.518665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b4745-fa34-4d21-93f8-b001edba3ad4 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:01:59.527162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b4745-fa34-4d21-93f8-b001edba3ad4 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:01:59.532084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92792e66-b7cc-4084-a0a8-0ebfb18e7661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:01:59.533182 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92792e66-b7cc-4084-a0a8-0ebfb18e7661 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:01:59.533948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92792e66-b7cc-4084-a0a8-0ebfb18e7661 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:01:59.534442 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92792e66-b7cc-4084-a0a8-0ebfb18e7661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:01:59.546283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92792e66-b7cc-4084-a0a8-0ebfb18e7661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:01:59.547756 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92792e66-b7cc-4084-a0a8-0ebfb18e7661 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:01:59.557201 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1139/2282] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:01:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:02:00.853122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=55 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:09.553175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:40.487865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb5062-88fa-42a2-a7a6-c2bf077a4dad None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:40.489344 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d4cb5062-88fa-42a2-a7a6-c2bf077a4dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:40.603343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb5062-88fa-42a2-a7a6-c2bf077a4dad None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:40.608510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36db7a9-7819-48db-8bf4-751c48d82409 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:40.609202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36db7a9-7819-48db-8bf4-751c48d82409 service placement] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:40.609482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36db7a9-7819-48db-8bf4-751c48d82409 service placement] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:40.609753 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36db7a9-7819-48db-8bf4-751c48d82409 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:40.611579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c36db7a9-7819-48db-8bf4-751c48d82409 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:40.624507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36db7a9-7819-48db-8bf4-751c48d82409 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:40.626287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36db7a9-7819-48db-8bf4-751c48d82409 service placement] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:40.640054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1144/2283] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:02:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:02:42.602595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f3f93-3027-4660-9564-8dbec5bbf7cd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:42.603898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f3f93-3027-4660-9564-8dbec5bbf7cd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:42.603898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f3f93-3027-4660-9564-8dbec5bbf7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:42.622794 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ab9f3f93-3027-4660-9564-8dbec5bbf7cd None None] MFA Rules not processed for user `9dc1b321e70f42e2be177e161486f997`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:42.628364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ab9f3f93-3027-4660-9564-8dbec5bbf7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:42.692545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1140/2284] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:02:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:44.724923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36db7a9-7819-48db-8bf4-751c48d82409 service placement] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:44.726458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c36db7a9-7819-48db-8bf4-751c48d82409 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:44.799998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36db7a9-7819-48db-8bf4-751c48d82409 service placement] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:44.804631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:44.805511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:44.805848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:44.806282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] PATH_INFO: `/v3/users/63768186635d4e4bbbb3efdd546cef0c` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:44.807761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=63768186635d4e4bbbb3efdd546cef0c)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:44.810356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:45.066037 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63768186635d4e4bbbb3efdd546cef0c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:45.074328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63768186635d4e4bbbb3efdd546cef0c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:45.075481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63768186635d4e4bbbb3efdd546cef0c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:45.089606 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:45.091081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63768186635d4e4bbbb3efdd546cef0c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:45.229566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63768186635d4e4bbbb3efdd546cef0c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:45.237569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1145/2285] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:02:44 2022] DELETE /identity/v3/users/63768186635d4e4bbbb3efdd546cef0c => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:45.242028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80711754-2380-4d6c-8234-d4d4ad01ff0b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:45.242494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80711754-2380-4d6c-8234-d4d4ad01ff0b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:45.242606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80711754-2380-4d6c-8234-d4d4ad01ff0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:45.264541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-80711754-2380-4d6c-8234-d4d4ad01ff0b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:45.264541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-80711754-2380-4d6c-8234-d4d4ad01ff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:45.332747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1141/2286] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:45.343432 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:45.350575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:45.407795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a545435-46e0-4e33-9752-78ede9b8c9fe admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:45.411832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64530852-f184-43c3-b4ad-e89ba046d6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:45.412488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64530852-f184-43c3-b4ad-e89ba046d6ce service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:45.412885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64530852-f184-43c3-b4ad-e89ba046d6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:45.413164 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64530852-f184-43c3-b4ad-e89ba046d6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:45.422942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64530852-f184-43c3-b4ad-e89ba046d6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:45.424390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64530852-f184-43c3-b4ad-e89ba046d6ce service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:45.435280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1146/2287] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:02:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:02:45.890152 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80711754-2380-4d6c-8234-d4d4ad01ff0b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:45.891617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-80711754-2380-4d6c-8234-d4d4ad01ff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:45.905167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80711754-2380-4d6c-8234-d4d4ad01ff0b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:45.910752 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:45.912424 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:45.913029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:45.913578 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea admin admin] PATH_INFO: `/v3/projects/559181a2d6954b9c9b3fc6fc228c0d1f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:45.923982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=559181a2d6954b9c9b3fc6fc228c0d1f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:45.926213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:46.116062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '559181a2d6954b9c9b3fc6fc228c0d1f', 'request_id': 'req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:46.248586 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '559181a2d6954b9c9b3fc6fc228c0d1f', 'request_id': 'req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:46.255750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '559181a2d6954b9c9b3fc6fc228c0d1f', 'request_id': 'req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:46.267341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea admin admin] The token cache is being invalidate because project 559181a2d6954b9c9b3fc6fc228c0d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:02:46.267628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:46.268308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6fdbc85b-7e58-49d1-97e4-e499d74c3aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:46.269873 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1142/2288] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:02:45 2022] DELETE /identity/v3/projects/559181a2d6954b9c9b3fc6fc228c0d1f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:46.365972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64530852-f184-43c3-b4ad-e89ba046d6ce service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:46.367538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-64530852-f184-43c3-b4ad-e89ba046d6ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:46.381275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64530852-f184-43c3-b4ad-e89ba046d6ce service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:46.457909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:46.459220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:46.459612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:46.459980 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] PATH_INFO: `/v3/users/3d27063d8dfd4c57823a7e40aa254906` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:46.461700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d27063d8dfd4c57823a7e40aa254906)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:46.463857 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:46.798741 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d27063d8dfd4c57823a7e40aa254906'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:46.818089 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d27063d8dfd4c57823a7e40aa254906'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:46.822795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d27063d8dfd4c57823a7e40aa254906'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:46.843682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d27063d8dfd4c57823a7e40aa254906'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:46.943837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d27063d8dfd4c57823a7e40aa254906'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:46.952716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1147/2289] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:02:46 2022] DELETE /identity/v3/users/3d27063d8dfd4c57823a7e40aa254906 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:46.957364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b34817-6405-4f7c-ac95-dfecbb18a686 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:46.957875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b34817-6405-4f7c-ac95-dfecbb18a686 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:46.959397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b34817-6405-4f7c-ac95-dfecbb18a686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:46.985220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b5b34817-6405-4f7c-ac95-dfecbb18a686 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:46.988136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b5b34817-6405-4f7c-ac95-dfecbb18a686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:47.016209 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:47.058171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1143/2290] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:47.072880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:47.075066 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:47.100481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:47.158526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98121db-d652-4882-b30f-df8b919fc017 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:47.162946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19743386-779c-4295-8d13-b7c3e6f6d893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:47.163730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19743386-779c-4295-8d13-b7c3e6f6d893 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:47.164016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19743386-779c-4295-8d13-b7c3e6f6d893 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:47.164291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19743386-779c-4295-8d13-b7c3e6f6d893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:47.176384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19743386-779c-4295-8d13-b7c3e6f6d893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:47.177830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19743386-779c-4295-8d13-b7c3e6f6d893 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:47.188901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1148/2291] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:02:48.042476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b34817-6405-4f7c-ac95-dfecbb18a686 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:48.046582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b5b34817-6405-4f7c-ac95-dfecbb18a686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:48.067472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b34817-6405-4f7c-ac95-dfecbb18a686 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:48.076551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:48.080878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:48.080878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:48.080878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] PATH_INFO: `/v3/projects/b155fe54367448d08baceb74fa8c1206` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:48.081930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b155fe54367448d08baceb74fa8c1206)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:48.085671 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:48.237988 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b155fe54367448d08baceb74fa8c1206', 'request_id': 'req-f0d659b1-e2e4-408d-b509-915947aac9bf'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:48.335196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b155fe54367448d08baceb74fa8c1206', 'request_id': 'req-f0d659b1-e2e4-408d-b509-915947aac9bf'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:48.342884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b155fe54367448d08baceb74fa8c1206', 'request_id': 'req-f0d659b1-e2e4-408d-b509-915947aac9bf'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:48.353309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] The token cache is being invalidate because project b155fe54367448d08baceb74fa8c1206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:02:48.353597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:48.354240 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:48.355897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1144/2292] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:02:48 2022] DELETE /identity/v3/projects/b155fe54367448d08baceb74fa8c1206 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:02:48.359838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19743386-779c-4295-8d13-b7c3e6f6d893 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:48.369631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19743386-779c-4295-8d13-b7c3e6f6d893 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:48.428950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c71a7-8d85-4378-a33e-859b5ce40107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:48.429947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c71a7-8d85-4378-a33e-859b5ce40107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:48.430246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c71a7-8d85-4378-a33e-859b5ce40107 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:48.430520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c71a7-8d85-4378-a33e-859b5ce40107 admin admin] PATH_INFO: `/v3/users/2abb242e351141fbb3e0e4117230ed0f` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:48.431799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c71a7-8d85-4378-a33e-859b5ce40107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2abb242e351141fbb3e0e4117230ed0f)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:48.434240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c71a7-8d85-4378-a33e-859b5ce40107 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:48.623742 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ce8c71a7-8d85-4378-a33e-859b5ce40107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2abb242e351141fbb3e0e4117230ed0f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:48.630869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ce8c71a7-8d85-4378-a33e-859b5ce40107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2abb242e351141fbb3e0e4117230ed0f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:48.631557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ce8c71a7-8d85-4378-a33e-859b5ce40107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2abb242e351141fbb3e0e4117230ed0f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:48.644496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ce8c71a7-8d85-4378-a33e-859b5ce40107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2abb242e351141fbb3e0e4117230ed0f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:48.722989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ce8c71a7-8d85-4378-a33e-859b5ce40107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2abb242e351141fbb3e0e4117230ed0f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:48.729876 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1149/2293] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:02:48 2022] DELETE /identity/v3/users/2abb242e351141fbb3e0e4117230ed0f => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:49.728041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:49.728041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:49.741364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d659b1-e2e4-408d-b509-915947aac9bf admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:49.834888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc71a5-de5f-4ec1-b833-41e0cc47357e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:49.836166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc71a5-de5f-4ec1-b833-41e0cc47357e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:49.836581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc71a5-de5f-4ec1-b833-41e0cc47357e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:49.836975 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc71a5-de5f-4ec1-b833-41e0cc47357e admin admin] PATH_INFO: `/v3/projects/1428cbfda9cb4b7ca83b5ffbf665909b` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:49.838644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc71a5-de5f-4ec1-b833-41e0cc47357e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1428cbfda9cb4b7ca83b5ffbf665909b)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:49.841686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc71a5-de5f-4ec1-b833-41e0cc47357e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:49.988804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-79dc71a5-de5f-4ec1-b833-41e0cc47357e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1428cbfda9cb4b7ca83b5ffbf665909b', 'request_id': 'req-79dc71a5-de5f-4ec1-b833-41e0cc47357e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:49.993905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:50.119318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-79dc71a5-de5f-4ec1-b833-41e0cc47357e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1428cbfda9cb4b7ca83b5ffbf665909b', 'request_id': 'req-79dc71a5-de5f-4ec1-b833-41e0cc47357e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:50.128516 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-79dc71a5-de5f-4ec1-b833-41e0cc47357e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1428cbfda9cb4b7ca83b5ffbf665909b', 'request_id': 'req-79dc71a5-de5f-4ec1-b833-41e0cc47357e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:50.141105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-79dc71a5-de5f-4ec1-b833-41e0cc47357e admin admin] The token cache is being invalidate because project 1428cbfda9cb4b7ca83b5ffbf665909b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:02:50.141386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-79dc71a5-de5f-4ec1-b833-41e0cc47357e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:50.142072 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-79dc71a5-de5f-4ec1-b833-41e0cc47357e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:50.143723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1145/2294] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:02:49 2022] DELETE /identity/v3/projects/1428cbfda9cb4b7ca83b5ffbf665909b => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:50.148590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447761a-b795-4297-b332-a1fb6db5ea6b None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.149088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447761a-b795-4297-b332-a1fb6db5ea6b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.149189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447761a-b795-4297-b332-a1fb6db5ea6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.171946 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-e447761a-b795-4297-b332-a1fb6db5ea6b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:50.173437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e447761a-b795-4297-b332-a1fb6db5ea6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:50.209818 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:50.245126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1150/2295] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:50.253931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd4359-04a0-4b95-b147-234beb5b0a58 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.254222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd4359-04a0-4b95-b147-234beb5b0a58 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.254499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd4359-04a0-4b95-b147-234beb5b0a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.275167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-62bd4359-04a0-4b95-b147-234beb5b0a58 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:50.276630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-62bd4359-04a0-4b95-b147-234beb5b0a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:50.287381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1146/2296] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:50.291804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447761a-b795-4297-b332-a1fb6db5ea6b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.302720 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447761a-b795-4297-b332-a1fb6db5ea6b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.307484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092370e-7418-4174-b336-f9a09c1c9c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.308147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092370e-7418-4174-b336-f9a09c1c9c5c admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.309036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092370e-7418-4174-b336-f9a09c1c9c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.309360 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092370e-7418-4174-b336-f9a09c1c9c5c admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.310096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092370e-7418-4174-b336-f9a09c1c9c5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:02:50.310399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092370e-7418-4174-b336-f9a09c1c9c5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:50.312040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092370e-7418-4174-b336-f9a09c1c9c5c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:50.326483 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1151/2297] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:50.331029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd4359-04a0-4b95-b147-234beb5b0a58 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.340903 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd4359-04a0-4b95-b147-234beb5b0a58 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.345804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d28d9f-d4ac-4408-b235-e863369650f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.346562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d28d9f-d4ac-4408-b235-e863369650f5 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.346766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d28d9f-d4ac-4408-b235-e863369650f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.347109 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d28d9f-d4ac-4408-b235-e863369650f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.347804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d28d9f-d4ac-4408-b235-e863369650f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:50.349699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d28d9f-d4ac-4408-b235-e863369650f5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:50.460084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1147/2298] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:02:50 2022] POST /identity/v3/projects => generated 378 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:02:50.461245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092370e-7418-4174-b336-f9a09c1c9c5c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.474258 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092370e-7418-4174-b336-f9a09c1c9c5c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.538291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f470c-528d-4916-91be-ebea9e000cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.539055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f470c-528d-4916-91be-ebea9e000cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.539972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f470c-528d-4916-91be-ebea9e000cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.539972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f470c-528d-4916-91be-ebea9e000cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.540293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f470c-528d-4916-91be-ebea9e000cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:50.542637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f470c-528d-4916-91be-ebea9e000cf7 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:50.639411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1152/2299] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:02:50 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:02:50.643296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d28d9f-d4ac-4408-b235-e863369650f5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.654427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d28d9f-d4ac-4408-b235-e863369650f5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.659180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973dea9-8785-49a2-abd6-f31290210ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.659876 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973dea9-8785-49a2-abd6-f31290210ded admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.660159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973dea9-8785-49a2-abd6-f31290210ded admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.660430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973dea9-8785-49a2-abd6-f31290210ded admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.661153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973dea9-8785-49a2-abd6-f31290210ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:50.663039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973dea9-8785-49a2-abd6-f31290210ded admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:50.674348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1148/2300] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:02:50.678419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f470c-528d-4916-91be-ebea9e000cf7 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.689761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f470c-528d-4916-91be-ebea9e000cf7 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.694733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d581d5b-807f-4f7d-9285-b10d005a6e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.697512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d581d5b-807f-4f7d-9285-b10d005a6e2e admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.697512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d581d5b-807f-4f7d-9285-b10d005a6e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.697512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d581d5b-807f-4f7d-9285-b10d005a6e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.697512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d581d5b-807f-4f7d-9285-b10d005a6e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:50.698471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d581d5b-807f-4f7d-9285-b10d005a6e2e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:50.709534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1153/2301] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:02:50 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:02:50.713869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973dea9-8785-49a2-abd6-f31290210ded admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:50.726194 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973dea9-8785-49a2-abd6-f31290210ded admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:50.729253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540d903-cfbb-464d-aad9-bb8c705714ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:50.730129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540d903-cfbb-464d-aad9-bb8c705714ea admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.730544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540d903-cfbb-464d-aad9-bb8c705714ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.731078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540d903-cfbb-464d-aad9-bb8c705714ea admin admin] PATH_INFO: `/v3/projects/afed133b2fc843dd99eef0fa25384bd5/users/a8eeecebd5d1461086a342294665eff7/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.745811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540d903-cfbb-464d-aad9-bb8c705714ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=afed133b2fc843dd99eef0fa25384bd5, user_id=a8eeecebd5d1461086a342294665eff7, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:50.749159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540d903-cfbb-464d-aad9-bb8c705714ea admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:50.846213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1149/2302] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:02:50 2022] PUT /identity/v3/projects/afed133b2fc843dd99eef0fa25384bd5/users/a8eeecebd5d1461086a342294665eff7/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:50.857428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b274548-6943-462b-bc43-12aeb9636ce8 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:50.857722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b274548-6943-462b-bc43-12aeb9636ce8 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:50.858001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b274548-6943-462b-bc43-12aeb9636ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:50.885208 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-6b274548-6943-462b-bc43-12aeb9636ce8 None None] MFA Rules not processed for user `a8eeecebd5d1461086a342294665eff7`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:50.886448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6b274548-6943-462b-bc43-12aeb9636ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:50.951074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1154/2303] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:02:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:50.955565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540d903-cfbb-464d-aad9-bb8c705714ea admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.019274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540d903-cfbb-464d-aad9-bb8c705714ea admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.024093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21fcaf-0b4f-404c-ac81-81ec0ad68c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.024876 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21fcaf-0b4f-404c-ac81-81ec0ad68c67 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.025164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21fcaf-0b4f-404c-ac81-81ec0ad68c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.025438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21fcaf-0b4f-404c-ac81-81ec0ad68c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.026120 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21fcaf-0b4f-404c-ac81-81ec0ad68c67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.028051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21fcaf-0b4f-404c-ac81-81ec0ad68c67 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.130720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1150/2304] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:02:50 2022] POST /identity/v3/projects => generated 380 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:02:51.134362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b274548-6943-462b-bc43-12aeb9636ce8 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.147100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b274548-6943-462b-bc43-12aeb9636ce8 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.176124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:51.207620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480a3db-4a0f-4c56-94c9-fbca4bfde451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.208607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480a3db-4a0f-4c56-94c9-fbca4bfde451 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.208915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480a3db-4a0f-4c56-94c9-fbca4bfde451 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.209320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480a3db-4a0f-4c56-94c9-fbca4bfde451 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.210420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480a3db-4a0f-4c56-94c9-fbca4bfde451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.212449 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480a3db-4a0f-4c56-94c9-fbca4bfde451 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.306425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1155/2305] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:02:51 2022] POST /identity/v3/users => generated 291 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:02:51.310775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21fcaf-0b4f-404c-ac81-81ec0ad68c67 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.321124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21fcaf-0b4f-404c-ac81-81ec0ad68c67 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.326581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcf4b2-70a6-4bae-af5a-e755ee44254b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.327277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcf4b2-70a6-4bae-af5a-e755ee44254b admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.327617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcf4b2-70a6-4bae-af5a-e755ee44254b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.327871 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcf4b2-70a6-4bae-af5a-e755ee44254b admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.328596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fcf4b2-70a6-4bae-af5a-e755ee44254b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.330203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fcf4b2-70a6-4bae-af5a-e755ee44254b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.341625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1151/2306] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:02:51 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:02:51.345643 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480a3db-4a0f-4c56-94c9-fbca4bfde451 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.356026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480a3db-4a0f-4c56-94c9-fbca4bfde451 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.360414 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43938f09-8dbe-4e57-b37f-c6b45c2cc65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.361072 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43938f09-8dbe-4e57-b37f-c6b45c2cc65b admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.362081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43938f09-8dbe-4e57-b37f-c6b45c2cc65b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.362081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43938f09-8dbe-4e57-b37f-c6b45c2cc65b admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.362334 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43938f09-8dbe-4e57-b37f-c6b45c2cc65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.363951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43938f09-8dbe-4e57-b37f-c6b45c2cc65b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.374504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1156/2307] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:02:51 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:02:51.378685 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcf4b2-70a6-4bae-af5a-e755ee44254b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.389577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcf4b2-70a6-4bae-af5a-e755ee44254b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.394132 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217678-b0a4-4dd3-9963-8c88aafe4218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.395044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217678-b0a4-4dd3-9963-8c88aafe4218 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.395322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217678-b0a4-4dd3-9963-8c88aafe4218 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.395592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217678-b0a4-4dd3-9963-8c88aafe4218 admin admin] PATH_INFO: `/v3/projects/0041b7738bce4f29b9a35666b7f67eff/users/e628404fdd51433c91036fc6c5519997/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.412002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87217678-b0a4-4dd3-9963-8c88aafe4218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0041b7738bce4f29b9a35666b7f67eff, user_id=e628404fdd51433c91036fc6c5519997, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.414743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87217678-b0a4-4dd3-9963-8c88aafe4218 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.506444 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1152/2308] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:02:51 2022] PUT /identity/v3/projects/0041b7738bce4f29b9a35666b7f67eff/users/e628404fdd51433c91036fc6c5519997/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:51.517781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0725d-f6d9-4808-abc5-0ae96cebd0e8 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.517998 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0725d-f6d9-4808-abc5-0ae96cebd0e8 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.518275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0725d-f6d9-4808-abc5-0ae96cebd0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.544437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-14c0725d-f6d9-4808-abc5-0ae96cebd0e8 None None] MFA Rules not processed for user `e628404fdd51433c91036fc6c5519997`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:51.545640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-14c0725d-f6d9-4808-abc5-0ae96cebd0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:51.613823 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1157/2309] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:02:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:51.618338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217678-b0a4-4dd3-9963-8c88aafe4218 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.651234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:51.682606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217678-b0a4-4dd3-9963-8c88aafe4218 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.688767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95d251-f652-4fba-8087-31d054704bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.689436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95d251-f652-4fba-8087-31d054704bca admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.689720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95d251-f652-4fba-8087-31d054704bca admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.689997 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95d251-f652-4fba-8087-31d054704bca admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.690687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95d251-f652-4fba-8087-31d054704bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.692723 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95d251-f652-4fba-8087-31d054704bca admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.812289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1153/2310] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:02:51 2022] POST /identity/v3/projects => generated 378 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:02:51.816341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0725d-f6d9-4808-abc5-0ae96cebd0e8 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.828995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0725d-f6d9-4808-abc5-0ae96cebd0e8 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:51.889684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dc903-78ad-4562-8f5a-78d85db2257d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:51.890359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dc903-78ad-4562-8f5a-78d85db2257d admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:51.890637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dc903-78ad-4562-8f5a-78d85db2257d admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:51.890950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dc903-78ad-4562-8f5a-78d85db2257d admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:51.891640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dc903-78ad-4562-8f5a-78d85db2257d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:51.893591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dc903-78ad-4562-8f5a-78d85db2257d admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:51.982153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1158/2311] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:02:51 2022] POST /identity/v3/users => generated 296 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:02:51.988019 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95d251-f652-4fba-8087-31d054704bca admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:51.999383 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95d251-f652-4fba-8087-31d054704bca admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:52.004170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e611d1-27cf-4d94-be0b-d95b3862be54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:52.004833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e611d1-27cf-4d94-be0b-d95b3862be54 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.005115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e611d1-27cf-4d94-be0b-d95b3862be54 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.005392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e611d1-27cf-4d94-be0b-d95b3862be54 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.006126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e611d1-27cf-4d94-be0b-d95b3862be54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:52.007827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e611d1-27cf-4d94-be0b-d95b3862be54 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:52.019318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1154/2312] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:02:51 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:02:52.024995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dc903-78ad-4562-8f5a-78d85db2257d admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:52.036780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dc903-78ad-4562-8f5a-78d85db2257d admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:52.044742 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f630412-69e9-404a-bcab-219c5ef5e678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:52.045671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f630412-69e9-404a-bcab-219c5ef5e678 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.045960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f630412-69e9-404a-bcab-219c5ef5e678 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.046235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f630412-69e9-404a-bcab-219c5ef5e678 admin admin] PATH_INFO: `/v3/projects/4f4a45e0e48748de913c7f7b3d410f8f/users/ddccf7648b75491b8de4beebb1b5ad70/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.063217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f630412-69e9-404a-bcab-219c5ef5e678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f4a45e0e48748de913c7f7b3d410f8f, user_id=ddccf7648b75491b8de4beebb1b5ad70, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:52.067906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f630412-69e9-404a-bcab-219c5ef5e678 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:52.131816 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1159/2313] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:02:52 2022] PUT /identity/v3/projects/4f4a45e0e48748de913c7f7b3d410f8f/users/ddccf7648b75491b8de4beebb1b5ad70/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:02:52.136008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e611d1-27cf-4d94-be0b-d95b3862be54 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:52.146999 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e611d1-27cf-4d94-be0b-d95b3862be54 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:52.206195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa570bf5-3bcb-4bb5-bef5-80a53568dc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:52.206906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa570bf5-3bcb-4bb5-bef5-80a53568dc55 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.207193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa570bf5-3bcb-4bb5-bef5-80a53568dc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.207454 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa570bf5-3bcb-4bb5-bef5-80a53568dc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.208175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa570bf5-3bcb-4bb5-bef5-80a53568dc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:52.209780 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa570bf5-3bcb-4bb5-bef5-80a53568dc55 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:52.220882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1155/2314] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:02:52 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:02:52.224957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f630412-69e9-404a-bcab-219c5ef5e678 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:52.231217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:52.236594 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f630412-69e9-404a-bcab-219c5ef5e678 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:52.248538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1475c3-ea88-471c-8097-8c28a9833d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:52.248538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1475c3-ea88-471c-8097-8c28a9833d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.248538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1475c3-ea88-471c-8097-8c28a9833d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.248538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1475c3-ea88-471c-8097-8c28a9833d84 admin admin] PATH_INFO: `/v3/projects/4f4a45e0e48748de913c7f7b3d410f8f/users/ddccf7648b75491b8de4beebb1b5ad70/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.249127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1475c3-ea88-471c-8097-8c28a9833d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f4a45e0e48748de913c7f7b3d410f8f, user_id=ddccf7648b75491b8de4beebb1b5ad70, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:52.249127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1475c3-ea88-471c-8097-8c28a9833d84 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:52.257850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1160/2315] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:02:52 2022] PUT /identity/v3/projects/4f4a45e0e48748de913c7f7b3d410f8f/users/ddccf7648b75491b8de4beebb1b5ad70/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:52.268243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240e170-507b-4bb1-9607-8595e1ade9f6 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.268533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240e170-507b-4bb1-9607-8595e1ade9f6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.268813 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240e170-507b-4bb1-9607-8595e1ade9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.297201 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6240e170-507b-4bb1-9607-8595e1ade9f6 None None] MFA Rules not processed for user `ddccf7648b75491b8de4beebb1b5ad70`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:52.298409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6240e170-507b-4bb1-9607-8595e1ade9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:52.371022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1156/2316] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:02:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:52.382422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1475c3-ea88-471c-8097-8c28a9833d84 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:52.384164 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-de1475c3-ea88-471c-8097-8c28a9833d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:52.443473 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1475c3-ea88-471c-8097-8c28a9833d84 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:52.447063 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a153d-8e1f-498c-80c7-453ca0f7ef22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:52.448664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a153d-8e1f-498c-80c7-453ca0f7ef22 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.448664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a153d-8e1f-498c-80c7-453ca0f7ef22 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.448664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a153d-8e1f-498c-80c7-453ca0f7ef22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.458528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a153d-8e1f-498c-80c7-453ca0f7ef22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:52.460190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a153d-8e1f-498c-80c7-453ca0f7ef22 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:52.470425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1161/2317] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:02:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:02:52.613968 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240e170-507b-4bb1-9607-8595e1ade9f6 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:52.615150 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6240e170-507b-4bb1-9607-8595e1ade9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:52.627227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240e170-507b-4bb1-9607-8595e1ade9f6 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:52.630633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e9226-d305-4724-aa35-7889469b3b43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:52.631304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e9226-d305-4724-aa35-7889469b3b43 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:52.631582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e9226-d305-4724-aa35-7889469b3b43 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:52.631849 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e9226-d305-4724-aa35-7889469b3b43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:52.653540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:52.690155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e9226-d305-4724-aa35-7889469b3b43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:52.691621 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e9226-d305-4724-aa35-7889469b3b43 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:52.701296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1157/2318] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:02:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:02:57.783250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a153d-8e1f-498c-80c7-453ca0f7ef22 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:57.854571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a153d-8e1f-498c-80c7-453ca0f7ef22 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:57.863040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:57.864068 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:57.864453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:57.864825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] PATH_INFO: `/v3/users/a8eeecebd5d1461086a342294665eff7` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:57.866323 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8eeecebd5d1461086a342294665eff7)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:57.868515 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:58.052480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8eeecebd5d1461086a342294665eff7'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:58.060159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8eeecebd5d1461086a342294665eff7'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:58.060840 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8eeecebd5d1461086a342294665eff7'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:58.072903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8eeecebd5d1461086a342294665eff7'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:58.131579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8eeecebd5d1461086a342294665eff7'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:58.138382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1162/2319] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:02:57 2022] DELETE /identity/v3/users/a8eeecebd5d1461086a342294665eff7 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:58.142955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ee822-21c0-4f82-a16d-9ee8bbe11573 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:58.143438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ee822-21c0-4f82-a16d-9ee8bbe11573 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:58.143534 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ee822-21c0-4f82-a16d-9ee8bbe11573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:58.162884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f34ee822-21c0-4f82-a16d-9ee8bbe11573 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:58.164094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f34ee822-21c0-4f82-a16d-9ee8bbe11573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:58.224676 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1158/2320] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:58.235340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:58.236747 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:58.250276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:58.295725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660db077-53a8-47c9-9f19-411ad07efd33 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:58.301151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337f594-ebc9-4287-b822-895f30a9b2bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:58.301151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337f594-ebc9-4287-b822-895f30a9b2bd service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:58.301151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337f594-ebc9-4287-b822-895f30a9b2bd service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:58.301151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337f594-ebc9-4287-b822-895f30a9b2bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:58.310189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337f594-ebc9-4287-b822-895f30a9b2bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:58.311651 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337f594-ebc9-4287-b822-895f30a9b2bd service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:58.322106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1163/2321] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:02:58.756241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ee822-21c0-4f82-a16d-9ee8bbe11573 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:58.757831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f34ee822-21c0-4f82-a16d-9ee8bbe11573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:58.782337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ee822-21c0-4f82-a16d-9ee8bbe11573 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:58.785971 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:58.787172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:58.787461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:58.787712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] PATH_INFO: `/v3/projects/afed133b2fc843dd99eef0fa25384bd5` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:58.788930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afed133b2fc843dd99eef0fa25384bd5)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:58.790843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:58.905346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afed133b2fc843dd99eef0fa25384bd5', 'request_id': 'req-db8f45f6-7025-4e0e-935d-463f33027166'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:58.994533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afed133b2fc843dd99eef0fa25384bd5', 'request_id': 'req-db8f45f6-7025-4e0e-935d-463f33027166'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.001547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afed133b2fc843dd99eef0fa25384bd5', 'request_id': 'req-db8f45f6-7025-4e0e-935d-463f33027166'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.012323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] The token cache is being invalidate because project afed133b2fc843dd99eef0fa25384bd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:02:59.012629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.013265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.014801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1159/2322] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:02:58 2022] DELETE /identity/v3/projects/afed133b2fc843dd99eef0fa25384bd5 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:59.018765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337f594-ebc9-4287-b822-895f30a9b2bd service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:59.029440 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337f594-ebc9-4287-b822-895f30a9b2bd service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:59.087286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:59.088274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:59.088555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:59.088891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] PATH_INFO: `/v3/users/e628404fdd51433c91036fc6c5519997` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:59.090996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e628404fdd51433c91036fc6c5519997)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:59.093014 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:59.271959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e628404fdd51433c91036fc6c5519997'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.280104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e628404fdd51433c91036fc6c5519997'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.280899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e628404fdd51433c91036fc6c5519997'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.293995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e628404fdd51433c91036fc6c5519997'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.348909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e628404fdd51433c91036fc6c5519997'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.355638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1164/2323] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:02:59 2022] DELETE /identity/v3/users/e628404fdd51433c91036fc6c5519997 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:02:59.730860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:59.733025 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:59.745403 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f45f6-7025-4e0e-935d-463f33027166 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:59.795110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:59.811425 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:59.813272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:59.813272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:59.813272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] PATH_INFO: `/v3/projects/0041b7738bce4f29b9a35666b7f67eff` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:59.814039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0041b7738bce4f29b9a35666b7f67eff)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:59.815969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:59.915033 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0041b7738bce4f29b9a35666b7f67eff', 'request_id': 'req-cd4ef52e-1b22-4362-abcb-fb4d29c67447'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.968942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0041b7738bce4f29b9a35666b7f67eff', 'request_id': 'req-cd4ef52e-1b22-4362-abcb-fb4d29c67447'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.979315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0041b7738bce4f29b9a35666b7f67eff', 'request_id': 'req-cd4ef52e-1b22-4362-abcb-fb4d29c67447'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.993735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] The token cache is being invalidate because project 0041b7738bce4f29b9a35666b7f67eff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:02:59.994030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.994696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:02:59.997520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1160/2324] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:02:59 2022] DELETE /identity/v3/projects/0041b7738bce4f29b9a35666b7f67eff => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:00.001313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:00.002801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:00.017769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da2b85-4f93-4f16-a32e-cd2768958fd3 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:00.049285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:00.076067 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525cfde-ca90-4a06-bae1-8f2d7cbac417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:00.076950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525cfde-ca90-4a06-bae1-8f2d7cbac417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:00.077219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525cfde-ca90-4a06-bae1-8f2d7cbac417 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:00.077511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525cfde-ca90-4a06-bae1-8f2d7cbac417 admin admin] PATH_INFO: `/v3/users/ddccf7648b75491b8de4beebb1b5ad70` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:00.078684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525cfde-ca90-4a06-bae1-8f2d7cbac417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddccf7648b75491b8de4beebb1b5ad70)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:00.080618 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525cfde-ca90-4a06-bae1-8f2d7cbac417 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:00.287343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a525cfde-ca90-4a06-bae1-8f2d7cbac417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddccf7648b75491b8de4beebb1b5ad70'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:00.295002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a525cfde-ca90-4a06-bae1-8f2d7cbac417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddccf7648b75491b8de4beebb1b5ad70'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:00.295752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a525cfde-ca90-4a06-bae1-8f2d7cbac417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddccf7648b75491b8de4beebb1b5ad70'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:00.308291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a525cfde-ca90-4a06-bae1-8f2d7cbac417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddccf7648b75491b8de4beebb1b5ad70'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:00.400140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-a525cfde-ca90-4a06-bae1-8f2d7cbac417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddccf7648b75491b8de4beebb1b5ad70'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:00.407938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1165/2325] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:03:00 2022] DELETE /identity/v3/users/ddccf7648b75491b8de4beebb1b5ad70 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:03:00.814396 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:00.816010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:00.829378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ef52e-1b22-4362-abcb-fb4d29c67447 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:00.904023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b2997-bce5-46e5-86d6-2cea289c85f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:00.904883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b2997-bce5-46e5-86d6-2cea289c85f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:00.905162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b2997-bce5-46e5-86d6-2cea289c85f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:00.905435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b2997-bce5-46e5-86d6-2cea289c85f2 admin admin] PATH_INFO: `/v3/projects/4f4a45e0e48748de913c7f7b3d410f8f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:00.906603 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b2997-bce5-46e5-86d6-2cea289c85f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f4a45e0e48748de913c7f7b3d410f8f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:00.908573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b2997-bce5-46e5-86d6-2cea289c85f2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:01.037923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-428b2997-bce5-46e5-86d6-2cea289c85f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f4a45e0e48748de913c7f7b3d410f8f', 'request_id': 'req-428b2997-bce5-46e5-86d6-2cea289c85f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:01.129485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-428b2997-bce5-46e5-86d6-2cea289c85f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f4a45e0e48748de913c7f7b3d410f8f', 'request_id': 'req-428b2997-bce5-46e5-86d6-2cea289c85f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:01.136409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-428b2997-bce5-46e5-86d6-2cea289c85f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f4a45e0e48748de913c7f7b3d410f8f', 'request_id': 'req-428b2997-bce5-46e5-86d6-2cea289c85f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:01.147388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-428b2997-bce5-46e5-86d6-2cea289c85f2 admin admin] The token cache is being invalidate because project 4f4a45e0e48748de913c7f7b3d410f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:03:01.147681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-428b2997-bce5-46e5-86d6-2cea289c85f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:01.148324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-428b2997-bce5-46e5-86d6-2cea289c85f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:01.149847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1161/2326] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:03:00 2022] DELETE /identity/v3/projects/4f4a45e0e48748de913c7f7b3d410f8f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:01.156355 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77bf3d-d684-4727-a4c5-7fb69d9141d3 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.156873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77bf3d-d684-4727-a4c5-7fb69d9141d3 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.156984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77bf3d-d684-4727-a4c5-7fb69d9141d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.176356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-0f77bf3d-d684-4727-a4c5-7fb69d9141d3 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:01.177546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0f77bf3d-d684-4727-a4c5-7fb69d9141d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:01.245279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1166/2327] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:03:01.256875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864389b-c8c4-4724-8bcc-13f02108159b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.257165 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864389b-c8c4-4724-8bcc-13f02108159b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.257447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864389b-c8c4-4724-8bcc-13f02108159b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.270448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:01.277017 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c864389b-c8c4-4724-8bcc-13f02108159b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:01.278934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c864389b-c8c4-4724-8bcc-13f02108159b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:01.287625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1162/2328] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:03:01.291889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77bf3d-d684-4727-a4c5-7fb69d9141d3 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:01.304377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77bf3d-d684-4727-a4c5-7fb69d9141d3 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:01.310165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1e8ff-de3a-4e4d-ad2d-02f2183ee960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:01.310881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1e8ff-de3a-4e4d-ad2d-02f2183ee960 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.311276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1e8ff-de3a-4e4d-ad2d-02f2183ee960 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.311484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1e8ff-de3a-4e4d-ad2d-02f2183ee960 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.312191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1e8ff-de3a-4e4d-ad2d-02f2183ee960 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:01.312522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1e8ff-de3a-4e4d-ad2d-02f2183ee960 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:01.314320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1e8ff-de3a-4e4d-ad2d-02f2183ee960 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:01.327965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1167/2329] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:03:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:03:01.331933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864389b-c8c4-4724-8bcc-13f02108159b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:01.343214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864389b-c8c4-4724-8bcc-13f02108159b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:01.348435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c387c1-abac-4b70-8297-31ae250133e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:01.349130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c387c1-abac-4b70-8297-31ae250133e7 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.349413 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c387c1-abac-4b70-8297-31ae250133e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.349683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c387c1-abac-4b70-8297-31ae250133e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.350365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c387c1-abac-4b70-8297-31ae250133e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:01.352517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c387c1-abac-4b70-8297-31ae250133e7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:01.454078 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1163/2330] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:03:01 2022] POST /identity/v3/projects => generated 370 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:01.458345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1e8ff-de3a-4e4d-ad2d-02f2183ee960 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:01.469426 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1e8ff-de3a-4e4d-ad2d-02f2183ee960 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:01.506576 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:01.525156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96ec09-49ed-43a9-b143-883680e58285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:01.525828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96ec09-49ed-43a9-b143-883680e58285 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.526113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96ec09-49ed-43a9-b143-883680e58285 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.526387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96ec09-49ed-43a9-b143-883680e58285 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.527113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96ec09-49ed-43a9-b143-883680e58285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:01.529788 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96ec09-49ed-43a9-b143-883680e58285 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:01.632298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1168/2331] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:03:01 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:01.636430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c387c1-abac-4b70-8297-31ae250133e7 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:01.647805 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c387c1-abac-4b70-8297-31ae250133e7 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:01.653727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f3d85-ea9e-4b84-8088-8810e247af9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:01.654392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f3d85-ea9e-4b84-8088-8810e247af9a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.654670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f3d85-ea9e-4b84-8088-8810e247af9a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.654984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f3d85-ea9e-4b84-8088-8810e247af9a admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.655712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f3d85-ea9e-4b84-8088-8810e247af9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:01.657301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f3d85-ea9e-4b84-8088-8810e247af9a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:01.667737 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1164/2332] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:03:01 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:03:01.671735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96ec09-49ed-43a9-b143-883680e58285 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:01.681648 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96ec09-49ed-43a9-b143-883680e58285 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:01.686374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aebbfe-b892-469b-a748-55d7d4bb5fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:01.687068 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aebbfe-b892-469b-a748-55d7d4bb5fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.687344 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aebbfe-b892-469b-a748-55d7d4bb5fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.687614 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aebbfe-b892-469b-a748-55d7d4bb5fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.688355 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aebbfe-b892-469b-a748-55d7d4bb5fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:01.689959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aebbfe-b892-469b-a748-55d7d4bb5fb6 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:01.701239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1169/2333] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:03:01 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:03:01.704915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f3d85-ea9e-4b84-8088-8810e247af9a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:01.715475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f3d85-ea9e-4b84-8088-8810e247af9a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:01.720714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18386bdf-b395-4945-832f-df8c44aa38cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:01.721580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18386bdf-b395-4945-832f-df8c44aa38cc admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.721863 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18386bdf-b395-4945-832f-df8c44aa38cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.722136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18386bdf-b395-4945-832f-df8c44aa38cc admin admin] PATH_INFO: `/v3/projects/8f775a05235c4435b312175eab3cb82b/users/2e16cdb9509b4c25b5b7e6b1e6fbe150/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.737815 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18386bdf-b395-4945-832f-df8c44aa38cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f775a05235c4435b312175eab3cb82b, user_id=2e16cdb9509b4c25b5b7e6b1e6fbe150, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:01.740493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18386bdf-b395-4945-832f-df8c44aa38cc admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:01.812303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1165/2334] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:03:01 2022] PUT /identity/v3/projects/8f775a05235c4435b312175eab3cb82b/users/2e16cdb9509b4c25b5b7e6b1e6fbe150/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:01.823576 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128539bf-3342-42d9-93ff-e777dd7c811c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:01.823864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128539bf-3342-42d9-93ff-e777dd7c811c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:01.824709 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128539bf-3342-42d9-93ff-e777dd7c811c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:01.851237 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-128539bf-3342-42d9-93ff-e777dd7c811c None None] MFA Rules not processed for user `2e16cdb9509b4c25b5b7e6b1e6fbe150`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:01.852434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-128539bf-3342-42d9-93ff-e777dd7c811c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:01.917047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1170/2335] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:03:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:03:01.929757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18386bdf-b395-4945-832f-df8c44aa38cc admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:01.931544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-18386bdf-b395-4945-832f-df8c44aa38cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:02.016467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18386bdf-b395-4945-832f-df8c44aa38cc admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:02.020154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119008a-d22e-4dda-91b3-3af5aa23fdc6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '683c54f489f440b9842cb94980214df1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:02.020801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119008a-d22e-4dda-91b3-3af5aa23fdc6 service glance] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.021080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119008a-d22e-4dda-91b3-3af5aa23fdc6 service glance] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.021348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119008a-d22e-4dda-91b3-3af5aa23fdc6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.033540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119008a-d22e-4dda-91b3-3af5aa23fdc6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:02.033540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119008a-d22e-4dda-91b3-3af5aa23fdc6 service glance] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:02.042507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1166/2336] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:03:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:03:11.914118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=48,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:12.038429 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=59,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:24.744671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf450bfb-37bb-4541-9b1e-9d9b6278a5b0 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:24.748017 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf450bfb-37bb-4541-9b1e-9d9b6278a5b0 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:24.748017 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf450bfb-37bb-4541-9b1e-9d9b6278a5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:24.794285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-bf450bfb-37bb-4541-9b1e-9d9b6278a5b0 None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:24.795502 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-bf450bfb-37bb-4541-9b1e-9d9b6278a5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:24.860634 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1171/2337] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 18:03:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:03:24.999208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119008a-d22e-4dda-91b3-3af5aa23fdc6 service glance] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:25.001332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c119008a-d22e-4dda-91b3-3af5aa23fdc6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:25.089882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119008a-d22e-4dda-91b3-3af5aa23fdc6 service glance] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:25.094973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2257f-254c-4526-8f50-16db36d9b377 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:25.095627 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2257f-254c-4526-8f50-16db36d9b377 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:25.095906 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2257f-254c-4526-8f50-16db36d9b377 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:25.096183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2257f-254c-4526-8f50-16db36d9b377 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:25.104778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2257f-254c-4526-8f50-16db36d9b377 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:25.106366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2257f-254c-4526-8f50-16db36d9b377 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:25.114576 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1167/2338] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:03:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:03:34.857372 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=40 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:35.111457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=41 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:42.211800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228474b9-2c5f-4814-b167-a1ad998d4821 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:42.212319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228474b9-2c5f-4814-b167-a1ad998d4821 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:42.212411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228474b9-2c5f-4814-b167-a1ad998d4821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:42.231253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-228474b9-2c5f-4814-b167-a1ad998d4821 None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:42.232455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-228474b9-2c5f-4814-b167-a1ad998d4821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:42.239650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1172/2339] 10.222.0.121 () {58 vars in 972 bytes} [Thu May 26 18:03:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:03:42.252322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2257f-254c-4526-8f50-16db36d9b377 service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:42.263138 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2257f-254c-4526-8f50-16db36d9b377 service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:42.269147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057582f8-194d-4793-98fa-f52f18010f02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:42.269794 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057582f8-194d-4793-98fa-f52f18010f02 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:42.270079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057582f8-194d-4793-98fa-f52f18010f02 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:42.270354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057582f8-194d-4793-98fa-f52f18010f02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:42.278772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057582f8-194d-4793-98fa-f52f18010f02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:42.280472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057582f8-194d-4793-98fa-f52f18010f02 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:42.290053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1168/2340] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:03:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:03:52.225825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:52.286385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:56.894285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228474b9-2c5f-4814-b167-a1ad998d4821 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:56.897085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-228474b9-2c5f-4814-b167-a1ad998d4821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:56.972497 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228474b9-2c5f-4814-b167-a1ad998d4821 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:56.977113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac84a0-856f-4471-a232-9c3997f7aac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:56.977994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac84a0-856f-4471-a232-9c3997f7aac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:56.978276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac84a0-856f-4471-a232-9c3997f7aac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:56.978551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac84a0-856f-4471-a232-9c3997f7aac0 admin admin] PATH_INFO: `/v3/users/9dc1b321e70f42e2be177e161486f997` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:56.979782 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac84a0-856f-4471-a232-9c3997f7aac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc1b321e70f42e2be177e161486f997)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:56.981759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac84a0-856f-4471-a232-9c3997f7aac0 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:57.186417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b3ac84a0-856f-4471-a232-9c3997f7aac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc1b321e70f42e2be177e161486f997'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:57.193533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b3ac84a0-856f-4471-a232-9c3997f7aac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc1b321e70f42e2be177e161486f997'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:57.194296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b3ac84a0-856f-4471-a232-9c3997f7aac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc1b321e70f42e2be177e161486f997'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:57.206510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b3ac84a0-856f-4471-a232-9c3997f7aac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc1b321e70f42e2be177e161486f997'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:57.287101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b3ac84a0-856f-4471-a232-9c3997f7aac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc1b321e70f42e2be177e161486f997'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:57.293978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1173/2341] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:03:56 2022] DELETE /identity/v3/users/9dc1b321e70f42e2be177e161486f997 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:57.544389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057582f8-194d-4793-98fa-f52f18010f02 service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:57.558129 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057582f8-194d-4793-98fa-f52f18010f02 service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:57.622406 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8bb074-b105-41c4-9622-8a8a7211eef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:57.623304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bb074-b105-41c4-9622-8a8a7211eef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:57.623582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bb074-b105-41c4-9622-8a8a7211eef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:57.623980 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bb074-b105-41c4-9622-8a8a7211eef6 admin admin] PATH_INFO: `/v3/projects/e358a1e391cf4ea18f18b10ed3585a91` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:57.625848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8bb074-b105-41c4-9622-8a8a7211eef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e358a1e391cf4ea18f18b10ed3585a91)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:57.627924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8bb074-b105-41c4-9622-8a8a7211eef6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:57.770561 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8f8bb074-b105-41c4-9622-8a8a7211eef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e358a1e391cf4ea18f18b10ed3585a91', 'request_id': 'req-8f8bb074-b105-41c4-9622-8a8a7211eef6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:57.873172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8f8bb074-b105-41c4-9622-8a8a7211eef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e358a1e391cf4ea18f18b10ed3585a91', 'request_id': 'req-8f8bb074-b105-41c4-9622-8a8a7211eef6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:57.880537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8f8bb074-b105-41c4-9622-8a8a7211eef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e358a1e391cf4ea18f18b10ed3585a91', 'request_id': 'req-8f8bb074-b105-41c4-9622-8a8a7211eef6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:57.892598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8f8bb074-b105-41c4-9622-8a8a7211eef6 admin admin] The token cache is being invalidate because project e358a1e391cf4ea18f18b10ed3585a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:03:57.892883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8f8bb074-b105-41c4-9622-8a8a7211eef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:57.893541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8f8bb074-b105-41c4-9622-8a8a7211eef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:03:57.895225 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1169/2342] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:03:57 2022] DELETE /identity/v3/projects/e358a1e391cf4ea18f18b10ed3585a91 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:57.900115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2e7f3-b18f-441c-bfc8-82cb269e788a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:57.900635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2e7f3-b18f-441c-bfc8-82cb269e788a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:57.900748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2e7f3-b18f-441c-bfc8-82cb269e788a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:57.919093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5ce2e7f3-b18f-441c-bfc8-82cb269e788a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:57.920294 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5ce2e7f3-b18f-441c-bfc8-82cb269e788a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:57.982338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1174/2343] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:03:57.990908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe4cf1-bae1-432e-b4ef-54fcb111f5a0 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:57.994323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe4cf1-bae1-432e-b4ef-54fcb111f5a0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:57.994323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe4cf1-bae1-432e-b4ef-54fcb111f5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.010411 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-23fe4cf1-bae1-432e-b4ef-54fcb111f5a0 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:58.011685 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-23fe4cf1-bae1-432e-b4ef-54fcb111f5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:58.019610 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1170/2344] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:03:58.023894 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2e7f3-b18f-441c-bfc8-82cb269e788a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.034047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2e7f3-b18f-441c-bfc8-82cb269e788a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.038638 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3fad5-fe91-43b6-9fa9-c1b096ed63ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.039323 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3fad5-fe91-43b6-9fa9-c1b096ed63ce admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.039645 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3fad5-fe91-43b6-9fa9-c1b096ed63ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.040193 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3fad5-fe91-43b6-9fa9-c1b096ed63ce admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.040912 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3fad5-fe91-43b6-9fa9-c1b096ed63ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:03:58.041220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3fad5-fe91-43b6-9fa9-c1b096ed63ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:58.042865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3fad5-fe91-43b6-9fa9-c1b096ed63ce admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:58.055114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1175/2345] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:03:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:03:58.059683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe4cf1-bae1-432e-b4ef-54fcb111f5a0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.069448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe4cf1-bae1-432e-b4ef-54fcb111f5a0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.074002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710dd25-0d54-4964-a62c-5d34fc91d7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.074743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710dd25-0d54-4964-a62c-5d34fc91d7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.075069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710dd25-0d54-4964-a62c-5d34fc91d7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.075350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710dd25-0d54-4964-a62c-5d34fc91d7e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.076043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710dd25-0d54-4964-a62c-5d34fc91d7e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:58.077951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710dd25-0d54-4964-a62c-5d34fc91d7e9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:58.157865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1171/2346] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:03:58 2022] POST /identity/v3/projects => generated 376 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:58.162347 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3fad5-fe91-43b6-9fa9-c1b096ed63ce admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.173468 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3fad5-fe91-43b6-9fa9-c1b096ed63ce admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.177045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:58.230544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33e2b3-9d4a-4602-aa19-59a2afbcc738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.231252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33e2b3-9d4a-4602-aa19-59a2afbcc738 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.231536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33e2b3-9d4a-4602-aa19-59a2afbcc738 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.231808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33e2b3-9d4a-4602-aa19-59a2afbcc738 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.232620 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33e2b3-9d4a-4602-aa19-59a2afbcc738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:58.235156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33e2b3-9d4a-4602-aa19-59a2afbcc738 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:58.342831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1176/2347] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:03:58 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:03:58.347953 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710dd25-0d54-4964-a62c-5d34fc91d7e9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.358533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710dd25-0d54-4964-a62c-5d34fc91d7e9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.364140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9365dc-1815-4569-af11-1b73868866be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.364831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9365dc-1815-4569-af11-1b73868866be admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.365113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9365dc-1815-4569-af11-1b73868866be admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.365387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9365dc-1815-4569-af11-1b73868866be admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.366108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9365dc-1815-4569-af11-1b73868866be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:58.367772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9365dc-1815-4569-af11-1b73868866be admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:58.378502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1172/2348] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:03:58 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:03:58.382569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33e2b3-9d4a-4602-aa19-59a2afbcc738 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.393225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33e2b3-9d4a-4602-aa19-59a2afbcc738 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.397883 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec65fc-4e49-4587-87ac-365f52207266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.398543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec65fc-4e49-4587-87ac-365f52207266 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.398849 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec65fc-4e49-4587-87ac-365f52207266 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.399137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec65fc-4e49-4587-87ac-365f52207266 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.399859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec65fc-4e49-4587-87ac-365f52207266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:58.401441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec65fc-4e49-4587-87ac-365f52207266 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:58.412061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1177/2349] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:03:58 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:03:58.416575 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9365dc-1815-4569-af11-1b73868866be admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.426610 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9365dc-1815-4569-af11-1b73868866be admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.431210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf656b83-17ad-4b38-a5fb-688c75482b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.433397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf656b83-17ad-4b38-a5fb-688c75482b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.433397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf656b83-17ad-4b38-a5fb-688c75482b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.433397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf656b83-17ad-4b38-a5fb-688c75482b6b admin admin] PATH_INFO: `/v3/projects/5f572f164c514a5d9ffd38f7a9be65d6/users/667709eac67f467498b2dbe47f2716b1/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.448831 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf656b83-17ad-4b38-a5fb-688c75482b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f572f164c514a5d9ffd38f7a9be65d6, user_id=667709eac67f467498b2dbe47f2716b1, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:58.451584 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf656b83-17ad-4b38-a5fb-688c75482b6b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:58.516933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1173/2350] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:03:58 2022] PUT /identity/v3/projects/5f572f164c514a5d9ffd38f7a9be65d6/users/667709eac67f467498b2dbe47f2716b1/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:03:58.580383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12237cda-111f-40d7-b787-5120694fe7b2 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.580672 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12237cda-111f-40d7-b787-5120694fe7b2 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.580954 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12237cda-111f-40d7-b787-5120694fe7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.609895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-12237cda-111f-40d7-b787-5120694fe7b2 None None] MFA Rules not processed for user `667709eac67f467498b2dbe47f2716b1`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:58.610927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-12237cda-111f-40d7-b787-5120694fe7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:58.675451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1178/2351] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:03:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:03:58.686064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf656b83-17ad-4b38-a5fb-688c75482b6b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:58.688060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cf656b83-17ad-4b38-a5fb-688c75482b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:58.719512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:58.746435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf656b83-17ad-4b38-a5fb-688c75482b6b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:58.750898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba85b8-1397-40a7-8ebf-0c271a8d21ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:58.751728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba85b8-1397-40a7-8ebf-0c271a8d21ac service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:58.752390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba85b8-1397-40a7-8ebf-0c271a8d21ac service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:58.752914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba85b8-1397-40a7-8ebf-0c271a8d21ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:58.762716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba85b8-1397-40a7-8ebf-0c271a8d21ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:58.764177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba85b8-1397-40a7-8ebf-0c271a8d21ac service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:58.774558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1174/2352] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:03:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:03.908361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b7615-128a-453e-8e85-7095c3be75e5 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:03.908894 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b7615-128a-453e-8e85-7095c3be75e5 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:03.909061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b7615-128a-453e-8e85-7095c3be75e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:03.926629 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-2a7b7615-128a-453e-8e85-7095c3be75e5 None None] MFA Rules not processed for user `99f2374b551741f4bc602debd1df336b`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:03.927854 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-2a7b7615-128a-453e-8e85-7095c3be75e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:03.983776 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1179/2353] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:04:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:08.839837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=14 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:13.976923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:17.573557 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba85b8-1397-40a7-8ebf-0c271a8d21ac service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:17.646694 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba85b8-1397-40a7-8ebf-0c271a8d21ac service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:17.652956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:17.653865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:17.654458 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:17.654458 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] PATH_INFO: `/v3/users/667709eac67f467498b2dbe47f2716b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:17.656131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=667709eac67f467498b2dbe47f2716b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:17.658017 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:17.918677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '667709eac67f467498b2dbe47f2716b1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:17.928533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '667709eac67f467498b2dbe47f2716b1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:17.929730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '667709eac67f467498b2dbe47f2716b1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:17.942933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '667709eac67f467498b2dbe47f2716b1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:18.020443 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '667709eac67f467498b2dbe47f2716b1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:18.028491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1175/2354] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:04:17 2022] DELETE /identity/v3/users/667709eac67f467498b2dbe47f2716b1 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:18.032708 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a547233-0ec1-43d1-9d85-f34a8740f362 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:18.033199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a547233-0ec1-43d1-9d85-f34a8740f362 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:18.033281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a547233-0ec1-43d1-9d85-f34a8740f362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:18.052798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-6a547233-0ec1-43d1-9d85-f34a8740f362 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:18.054026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6a547233-0ec1-43d1-9d85-f34a8740f362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:18.123676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1180/2355] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:18.134041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:18.139327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:18.199779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17496075-eb7d-44da-a029-e4eb976079b9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:18.203609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e757ba1-8f23-4ccb-85b9-0b65237c1b4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:18.204265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e757ba1-8f23-4ccb-85b9-0b65237c1b4c service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:18.204542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e757ba1-8f23-4ccb-85b9-0b65237c1b4c service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:18.204915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e757ba1-8f23-4ccb-85b9-0b65237c1b4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:18.215164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e757ba1-8f23-4ccb-85b9-0b65237c1b4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:18.216601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e757ba1-8f23-4ccb-85b9-0b65237c1b4c service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:18.226986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1176/2356] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:18.818930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9b0e6-d28b-48c8-950f-da1292ca6d34 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:18.819635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9b0e6-d28b-48c8-950f-da1292ca6d34 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:18.820025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9b0e6-d28b-48c8-950f-da1292ca6d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:18.843592 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-82e9b0e6-d28b-48c8-950f-da1292ca6d34 None None] MFA Rules not processed for user `4ffcb99923b346938eb72c0b2786e815`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:18.845129 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-82e9b0e6-d28b-48c8-950f-da1292ca6d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:18.910359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1181/2357] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:04:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:18.923293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e757ba1-8f23-4ccb-85b9-0b65237c1b4c service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:18.924714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8e757ba1-8f23-4ccb-85b9-0b65237c1b4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:18.939732 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e757ba1-8f23-4ccb-85b9-0b65237c1b4c service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:18.945347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:18.946231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:18.946512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:18.946788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3 admin admin] PATH_INFO: `/v3/projects/5f572f164c514a5d9ffd38f7a9be65d6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:18.948333 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f572f164c514a5d9ffd38f7a9be65d6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:18.950337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:18.969402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:19.084601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f572f164c514a5d9ffd38f7a9be65d6', 'request_id': 'req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:19.167747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f572f164c514a5d9ffd38f7a9be65d6', 'request_id': 'req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:19.174994 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f572f164c514a5d9ffd38f7a9be65d6', 'request_id': 'req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:19.185960 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3 admin admin] The token cache is being invalidate because project 5f572f164c514a5d9ffd38f7a9be65d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:04:19.186281 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:19.187126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c6ad5a96-c6c0-456f-97a6-6db936aa24d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:19.189148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1177/2358] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:04:18 2022] DELETE /identity/v3/projects/5f572f164c514a5d9ffd38f7a9be65d6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:21.127692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9b0e6-d28b-48c8-950f-da1292ca6d34 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:21.129199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-82e9b0e6-d28b-48c8-950f-da1292ca6d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:21.146615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9b0e6-d28b-48c8-950f-da1292ca6d34 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:21.208994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:21.209917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:21.210198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:21.210471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] PATH_INFO: `/v3/users/2e16cdb9509b4c25b5b7e6b1e6fbe150` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:21.211744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e16cdb9509b4c25b5b7e6b1e6fbe150)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:21.213692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:21.438484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e16cdb9509b4c25b5b7e6b1e6fbe150'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:21.439744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:21.446330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e16cdb9509b4c25b5b7e6b1e6fbe150'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:21.447021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e16cdb9509b4c25b5b7e6b1e6fbe150'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:21.458352 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e16cdb9509b4c25b5b7e6b1e6fbe150'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:21.532933 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e16cdb9509b4c25b5b7e6b1e6fbe150'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:21.539863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1182/2359] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:04:21 2022] DELETE /identity/v3/users/2e16cdb9509b4c25b5b7e6b1e6fbe150 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:21.545017 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9ce40-c315-45cf-823e-02942287c170 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:21.545536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9ce40-c315-45cf-823e-02942287c170 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:21.545631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9ce40-c315-45cf-823e-02942287c170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:21.563763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f7f9ce40-c315-45cf-823e-02942287c170 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:21.565020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f7f9ce40-c315-45cf-823e-02942287c170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:21.624591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1178/2360] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:21.634396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:21.635786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:21.695413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a09d2b-dc14-417b-a360-195c94ee3d22 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:21.699074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c616e9-22ca-4873-aacb-972440ce3bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:21.699753 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c616e9-22ca-4873-aacb-972440ce3bca service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:21.700034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c616e9-22ca-4873-aacb-972440ce3bca service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:21.700306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c616e9-22ca-4873-aacb-972440ce3bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:21.709833 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c616e9-22ca-4873-aacb-972440ce3bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:21.716123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c616e9-22ca-4873-aacb-972440ce3bca service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:21.721438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1183/2361] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:22.341507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9ce40-c315-45cf-823e-02942287c170 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:22.342610 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f7f9ce40-c315-45cf-823e-02942287c170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:22.361333 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9ce40-c315-45cf-823e-02942287c170 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:22.367368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:22.368259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:22.368538 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:22.368811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] PATH_INFO: `/v3/projects/8f775a05235c4435b312175eab3cb82b` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:22.370051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f775a05235c4435b312175eab3cb82b)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:22.372019 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:22.500702 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f775a05235c4435b312175eab3cb82b', 'request_id': 'req-07f8c730-96aa-4daa-8966-fec3044fa747'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:22.573367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f775a05235c4435b312175eab3cb82b', 'request_id': 'req-07f8c730-96aa-4daa-8966-fec3044fa747'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:22.580668 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f775a05235c4435b312175eab3cb82b', 'request_id': 'req-07f8c730-96aa-4daa-8966-fec3044fa747'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:22.591299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] The token cache is being invalidate because project 8f775a05235c4435b312175eab3cb82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:04:22.591588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:22.592227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:22.593986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1179/2362] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:04:22 2022] DELETE /identity/v3/projects/8f775a05235c4435b312175eab3cb82b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:29.476511 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2baf8-de56-457d-9bab-1bfad4f45161 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:29.477459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2baf8-de56-457d-9bab-1bfad4f45161 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:29.477459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2baf8-de56-457d-9bab-1bfad4f45161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:29.495763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d9d2baf8-de56-457d-9bab-1bfad4f45161 None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:29.496984 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d9d2baf8-de56-457d-9bab-1bfad4f45161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:29.559608 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1184/2363] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 18:04:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:29.581234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:29.582626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:29.616450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:29.653349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8c730-96aa-4daa-8966-fec3044fa747 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:29.658407 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303aa2b-8a55-4365-8607-7438d07f96a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:29.659097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303aa2b-8a55-4365-8607-7438d07f96a6 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:29.659388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303aa2b-8a55-4365-8607-7438d07f96a6 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:29.659664 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303aa2b-8a55-4365-8607-7438d07f96a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:29.669853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303aa2b-8a55-4365-8607-7438d07f96a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:29.671484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303aa2b-8a55-4365-8607-7438d07f96a6 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:29.679747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1180/2364] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:04:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:04:39.414075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2baf8-de56-457d-9bab-1bfad4f45161 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.415617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d9d2baf8-de56-457d-9bab-1bfad4f45161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:39.480868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2baf8-de56-457d-9bab-1bfad4f45161 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.484349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e75cca-935e-4382-a66d-a5c1f66bc1db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.484991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e75cca-935e-4382-a66d-a5c1f66bc1db service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.485269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e75cca-935e-4382-a66d-a5c1f66bc1db service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.485536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e75cca-935e-4382-a66d-a5c1f66bc1db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.487341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-37e75cca-935e-4382-a66d-a5c1f66bc1db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:39.516357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:39.561125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e75cca-935e-4382-a66d-a5c1f66bc1db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.562562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e75cca-935e-4382-a66d-a5c1f66bc1db service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.573493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1185/2365] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:39.921970 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=19 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:40.754655 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303aa2b-8a55-4365-8607-7438d07f96a6 service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.765619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303aa2b-8a55-4365-8607-7438d07f96a6 service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.769842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9d4fe-d300-4f9c-bcfc-514b891cb3e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.776124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9d4fe-d300-4f9c-bcfc-514b891cb3e6 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.776124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9d4fe-d300-4f9c-bcfc-514b891cb3e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.776124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9d4fe-d300-4f9c-bcfc-514b891cb3e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.776124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-20f9d4fe-d300-4f9c-bcfc-514b891cb3e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:40.787313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f9d4fe-d300-4f9c-bcfc-514b891cb3e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.788765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f9d4fe-d300-4f9c-bcfc-514b891cb3e6 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.800804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1181/2366] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:04:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:49.163639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e75cca-935e-4382-a66d-a5c1f66bc1db service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:49.166089 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-37e75cca-935e-4382-a66d-a5c1f66bc1db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:49.180982 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e75cca-935e-4382-a66d-a5c1f66bc1db service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:49.186949 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da3d21-54fc-49a0-ac29-521be3766662 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:49.187599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da3d21-54fc-49a0-ac29-521be3766662 service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:49.187909 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da3d21-54fc-49a0-ac29-521be3766662 service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:49.188184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da3d21-54fc-49a0-ac29-521be3766662 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:49.189940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-44da3d21-54fc-49a0-ac29-521be3766662 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:49.249947 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da3d21-54fc-49a0-ac29-521be3766662 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:49.251596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da3d21-54fc-49a0-ac29-521be3766662 service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:49.261569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1186/2367] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:04:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:50.795615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:53.168452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9d4fe-d300-4f9c-bcfc-514b891cb3e6 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:53.177219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9d4fe-d300-4f9c-bcfc-514b891cb3e6 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:53.181063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a37e6d-cdda-4773-b953-dd2b691706f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:53.181701 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a37e6d-cdda-4773-b953-dd2b691706f4 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.181982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a37e6d-cdda-4773-b953-dd2b691706f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.182304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a37e6d-cdda-4773-b953-dd2b691706f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.184865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d8a37e6d-cdda-4773-b953-dd2b691706f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:53.194481 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a37e6d-cdda-4773-b953-dd2b691706f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:53.196573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a37e6d-cdda-4773-b953-dd2b691706f4 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:53.204448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1182/2368] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:04:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:53.531113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c7afd-c2e4-45b4-b390-c0a83bc6cd7c None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.531657 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c7afd-c2e4-45b4-b390-c0a83bc6cd7c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.531657 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c7afd-c2e4-45b4-b390-c0a83bc6cd7c None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.533097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1187/2369] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 18:04:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 18:04:53.538754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d888a3-d8c9-40f5-afef-53a4faad52a5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.539074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d888a3-d8c9-40f5-afef-53a4faad52a5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.539404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d888a3-d8c9-40f5-afef-53a4faad52a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.577788 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-02d888a3-d8c9-40f5-afef-53a4faad52a5 None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:53.579086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-02d888a3-d8c9-40f5-afef-53a4faad52a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:53.589376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1183/2370] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 18:04:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:53.600885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c7afd-c2e4-45b4-b390-c0a83bc6cd7c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:53.602740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c69c7afd-c2e4-45b4-b390-c0a83bc6cd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:53.661463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c7afd-c2e4-45b4-b390-c0a83bc6cd7c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:53.665029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc39258-b436-4905-adaf-ef9e4fdbfd88 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '683c54f489f440b9842cb94980214df1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:53.665665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc39258-b436-4905-adaf-ef9e4fdbfd88 service glance] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.665943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc39258-b436-4905-adaf-ef9e4fdbfd88 service glance] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.666214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc39258-b436-4905-adaf-ef9e4fdbfd88 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.676825 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc39258-b436-4905-adaf-ef9e4fdbfd88 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:53.678241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc39258-b436-4905-adaf-ef9e4fdbfd88 service glance] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:53.689364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1188/2371] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:04:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:53.947193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d102d-b366-4fa6-9cef-ea79e72f0bbe None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.947566 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d102d-b366-4fa6-9cef-ea79e72f0bbe None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.947905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d102d-b366-4fa6-9cef-ea79e72f0bbe None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.949416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1184/2372] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 18:04:53 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 18:04:53.954437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10029903-eef1-43c1-9e09-6aedbbb62f68 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.954729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10029903-eef1-43c1-9e09-6aedbbb62f68 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.955069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10029903-eef1-43c1-9e09-6aedbbb62f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.973605 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-10029903-eef1-43c1-9e09-6aedbbb62f68 None None] MFA Rules not processed for user `08709b462300437598c8db331e5ad20f`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:53.974813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-10029903-eef1-43c1-9e09-6aedbbb62f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:53.981912 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1189/2373] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 18:04:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:53.993487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d102d-b366-4fa6-9cef-ea79e72f0bbe None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:53.995841 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e64d102d-b366-4fa6-9cef-ea79e72f0bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.009330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d102d-b366-4fa6-9cef-ea79e72f0bbe None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.012856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63634a-54b6-4fe7-ab57-9ffb5dcfed12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.013493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63634a-54b6-4fe7-ab57-9ffb5dcfed12 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.013772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63634a-54b6-4fe7-ab57-9ffb5dcfed12 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.014044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63634a-54b6-4fe7-ab57-9ffb5dcfed12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.021052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63634a-54b6-4fe7-ab57-9ffb5dcfed12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.022469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63634a-54b6-4fe7-ab57-9ffb5dcfed12 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.030482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1185/2374] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:04:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:54.117268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c7b8c-23e0-479d-b0f5-a2140c1ea5f6 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.117557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c7b8c-23e0-479d-b0f5-a2140c1ea5f6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.117831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c7b8c-23e0-479d-b0f5-a2140c1ea5f6 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.119412 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1190/2375] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 18:04:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 18:04:54.124779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34de57-4616-4b44-aff8-019a634d38fd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.125061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34de57-4616-4b44-aff8-019a634d38fd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.125337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34de57-4616-4b44-aff8-019a634d38fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.144983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6f34de57-4616-4b44-aff8-019a634d38fd None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:54.146218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6f34de57-4616-4b44-aff8-019a634d38fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.156940 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1186/2376] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 18:04:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:54.169089 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c7b8c-23e0-479d-b0f5-a2140c1ea5f6 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.180852 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c7b8c-23e0-479d-b0f5-a2140c1ea5f6 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.184868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a229104-801e-4de3-a293-8144858636aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.185536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a229104-801e-4de3-a293-8144858636aa service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.185814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a229104-801e-4de3-a293-8144858636aa service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.186086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a229104-801e-4de3-a293-8144858636aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.198160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a229104-801e-4de3-a293-8144858636aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.199566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a229104-801e-4de3-a293-8144858636aa service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.211722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1191/2377] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:04:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:54.287959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34de57-4616-4b44-aff8-019a634d38fd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.289536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6f34de57-4616-4b44-aff8-019a634d38fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.359851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34de57-4616-4b44-aff8-019a634d38fd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.364913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45086c45-f8af-4fe2-b7f9-d0ad7cc7a4e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.365564 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45086c45-f8af-4fe2-b7f9-d0ad7cc7a4e0 service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.365841 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45086c45-f8af-4fe2-b7f9-d0ad7cc7a4e0 service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.366116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45086c45-f8af-4fe2-b7f9-d0ad7cc7a4e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.367942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-45086c45-f8af-4fe2-b7f9-d0ad7cc7a4e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.380074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45086c45-f8af-4fe2-b7f9-d0ad7cc7a4e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.381680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45086c45-f8af-4fe2-b7f9-d0ad7cc7a4e0 service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.392373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1187/2378] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:04:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:58.074655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a229104-801e-4de3-a293-8144858636aa service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:58.083905 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a229104-801e-4de3-a293-8144858636aa service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.089125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbaa0f9-ea37-4466-b71f-b23982ac77b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.089247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbaa0f9-ea37-4466-b71f-b23982ac77b1 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.089439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbaa0f9-ea37-4466-b71f-b23982ac77b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.089719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbaa0f9-ea37-4466-b71f-b23982ac77b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.091685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ebbaa0f9-ea37-4466-b71f-b23982ac77b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:58.102248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbaa0f9-ea37-4466-b71f-b23982ac77b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.104678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbaa0f9-ea37-4466-b71f-b23982ac77b1 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:58.114403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1192/2379] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:04:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:04:59.036198 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226d6f6-353a-4cde-bba6-364db19a42ff None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:59.036695 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226d6f6-353a-4cde-bba6-364db19a42ff None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:59.036695 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226d6f6-353a-4cde-bba6-364db19a42ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:59.054226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5226d6f6-353a-4cde-bba6-364db19a42ff None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:59.055459 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5226d6f6-353a-4cde-bba6-364db19a42ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:59.062766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1188/2380] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 18:04:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:59.077039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbaa0f9-ea37-4466-b71f-b23982ac77b1 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:59.091189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbaa0f9-ea37-4466-b71f-b23982ac77b1 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:59.096211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdafbc7-ab67-4a4b-8c0d-245ee199507b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:59.097495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdafbc7-ab67-4a4b-8c0d-245ee199507b service swift] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:59.097495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdafbc7-ab67-4a4b-8c0d-245ee199507b service swift] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:59.097495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdafbc7-ab67-4a4b-8c0d-245ee199507b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:59.106973 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdafbc7-ab67-4a4b-8c0d-245ee199507b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:59.108680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdafbc7-ab67-4a4b-8c0d-245ee199507b service swift] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:59.117601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1193/2381] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:04:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:05:09.049492 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=59 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:09.114526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:16.918424 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226d6f6-353a-4cde-bba6-364db19a42ff None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:16.938067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226d6f6-353a-4cde-bba6-364db19a42ff None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:16.939250 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e3826-3a7f-4e8c-a2fa-d31c55850d6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:16.939994 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e3826-3a7f-4e8c-a2fa-d31c55850d6b service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:16.940341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e3826-3a7f-4e8c-a2fa-d31c55850d6b service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:16.940681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e3826-3a7f-4e8c-a2fa-d31c55850d6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:16.942954 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d83e3826-3a7f-4e8c-a2fa-d31c55850d6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:16.956180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e3826-3a7f-4e8c-a2fa-d31c55850d6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:16.957762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e3826-3a7f-4e8c-a2fa-d31c55850d6b service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:16.969762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1189/2382] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:05:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:26.958537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdafbc7-ab67-4a4b-8c0d-245ee199507b service swift] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:26.965248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:26.972340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdafbc7-ab67-4a4b-8c0d-245ee199507b service swift] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:26.980992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178c03e-70a4-46b1-adfb-c1b8c5243dcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:26.980992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178c03e-70a4-46b1-adfb-c1b8c5243dcd service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:26.980992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178c03e-70a4-46b1-adfb-c1b8c5243dcd service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:26.980992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178c03e-70a4-46b1-adfb-c1b8c5243dcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:26.982810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4178c03e-70a4-46b1-adfb-c1b8c5243dcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:27.055939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178c03e-70a4-46b1-adfb-c1b8c5243dcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:27.055939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178c03e-70a4-46b1-adfb-c1b8c5243dcd service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:27.063125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1194/2383] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:05:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:28.795711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e3826-3a7f-4e8c-a2fa-d31c55850d6b service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:28.797521 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d83e3826-3a7f-4e8c-a2fa-d31c55850d6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:28.813381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e3826-3a7f-4e8c-a2fa-d31c55850d6b service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:28.819283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cab3c-3395-43a2-8884-5496bb356960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:28.819943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cab3c-3395-43a2-8884-5496bb356960 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:28.820291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cab3c-3395-43a2-8884-5496bb356960 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:28.820576 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cab3c-3395-43a2-8884-5496bb356960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:28.822380 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7f6cab3c-3395-43a2-8884-5496bb356960 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:28.835771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cab3c-3395-43a2-8884-5496bb356960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:28.837472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cab3c-3395-43a2-8884-5496bb356960 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:28.849783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1190/2384] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:05:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:31.550711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1623140-2564-419e-9430-169401d70147 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.551240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1623140-2564-419e-9430-169401d70147 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.551338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1623140-2564-419e-9430-169401d70147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.568994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c1623140-2564-419e-9430-169401d70147 None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:31.570694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c1623140-2564-419e-9430-169401d70147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:31.577428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1195/2385] 10.222.0.121 () {58 vars in 972 bytes} [Thu May 26 18:05:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:31.610128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cab3c-3395-43a2-8884-5496bb356960 service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:31.616024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cab3c-3395-43a2-8884-5496bb356960 service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:31.622632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a082998-2764-4267-a820-39333467dac6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:31.623498 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a082998-2764-4267-a820-39333467dac6 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.624213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a082998-2764-4267-a820-39333467dac6 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.624213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a082998-2764-4267-a820-39333467dac6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.634713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a082998-2764-4267-a820-39333467dac6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:31.636644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a082998-2764-4267-a820-39333467dac6 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:31.646695 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1191/2386] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:05:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:05:41.563928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=31 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:41.642755 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=12 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:50.497920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1623140-2564-419e-9430-169401d70147 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:50.499451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c1623140-2564-419e-9430-169401d70147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:50.572526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1623140-2564-419e-9430-169401d70147 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:50.577108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184e8d8-9918-46ee-a7d3-63c0ba144d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:50.577923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184e8d8-9918-46ee-a7d3-63c0ba144d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:50.578319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184e8d8-9918-46ee-a7d3-63c0ba144d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:50.578625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184e8d8-9918-46ee-a7d3-63c0ba144d7a admin admin] PATH_INFO: `/v3/users/99f2374b551741f4bc602debd1df336b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:50.579881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184e8d8-9918-46ee-a7d3-63c0ba144d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f2374b551741f4bc602debd1df336b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:50.581841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184e8d8-9918-46ee-a7d3-63c0ba144d7a admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:50.776799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6184e8d8-9918-46ee-a7d3-63c0ba144d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f2374b551741f4bc602debd1df336b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:50.784881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6184e8d8-9918-46ee-a7d3-63c0ba144d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f2374b551741f4bc602debd1df336b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:50.785581 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6184e8d8-9918-46ee-a7d3-63c0ba144d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f2374b551741f4bc602debd1df336b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:50.798469 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6184e8d8-9918-46ee-a7d3-63c0ba144d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f2374b551741f4bc602debd1df336b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:50.904246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6184e8d8-9918-46ee-a7d3-63c0ba144d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f2374b551741f4bc602debd1df336b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:50.911932 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1196/2387] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:05:50 2022] DELETE /identity/v3/users/99f2374b551741f4bc602debd1df336b => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:51.199185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a082998-2764-4267-a820-39333467dac6 service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:51.211488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a082998-2764-4267-a820-39333467dac6 service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:51.268464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:51.269324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:51.269703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:51.269983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] PATH_INFO: `/v3/projects/ed709461c5a0499abc1a9cc33a8f916d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:51.271174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed709461c5a0499abc1a9cc33a8f916d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:51.273153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:51.396816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed709461c5a0499abc1a9cc33a8f916d', 'request_id': 'req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:51.483525 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed709461c5a0499abc1a9cc33a8f916d', 'request_id': 'req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:51.490206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed709461c5a0499abc1a9cc33a8f916d', 'request_id': 'req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:51.501037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] The token cache is being invalidate because project ed709461c5a0499abc1a9cc33a8f916d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:51.501331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:51.501993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:51.503581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1192/2388] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:05:51 2022] DELETE /identity/v3/projects/ed709461c5a0499abc1a9cc33a8f916d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:00.526222 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0700f64-4005-4ebf-808a-814fb538d52a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:00.526753 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0700f64-4005-4ebf-808a-814fb538d52a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:00.526753 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0700f64-4005-4ebf-808a-814fb538d52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:00.544271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f0700f64-4005-4ebf-808a-814fb538d52a None None] MFA Rules not processed for user `dbee2da50ee6413db771071900703e29`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:00.545792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f0700f64-4005-4ebf-808a-814fb538d52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:00.600338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1197/2389] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:06:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:01.497542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:10.593612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:23.074798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:23.076356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:23.150675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1abb5-9f01-41cc-8c09-2beb9aaee42e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:23.156272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35daa9aa-2540-4838-a825-beb1c7d360a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:23.156921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daa9aa-2540-4838-a825-beb1c7d360a6 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:23.157205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daa9aa-2540-4838-a825-beb1c7d360a6 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:23.157483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daa9aa-2540-4838-a825-beb1c7d360a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:23.159299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-35daa9aa-2540-4838-a825-beb1c7d360a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:23.218506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35daa9aa-2540-4838-a825-beb1c7d360a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:23.220135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35daa9aa-2540-4838-a825-beb1c7d360a6 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:23.229103 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1193/2390] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:06:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:33.224881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=56 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:39.323311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c3534-4950-441f-b694-472bd8ca7949 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:39.323871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c3534-4950-441f-b694-472bd8ca7949 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:39.323871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c3534-4950-441f-b694-472bd8ca7949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:39.342280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-9b4c3534-4950-441f-b694-472bd8ca7949 None None] MFA Rules not processed for user `4ffcb99923b346938eb72c0b2786e815`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:39.343583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9b4c3534-4950-441f-b694-472bd8ca7949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:39.351072 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1198/2391] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:06:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:49.336528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:50.138167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6bdd2-b324-4178-beed-d6f905a7cc4c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:50.138702 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6bdd2-b324-4178-beed-d6f905a7cc4c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:50.138702 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6bdd2-b324-4178-beed-d6f905a7cc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:50.155985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-90a6bdd2-b324-4178-beed-d6f905a7cc4c None None] MFA Rules not processed for user `fe1c86c411834f1c8d2775a5b2c5f1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:50.157243 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-90a6bdd2-b324-4178-beed-d6f905a7cc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:50.212840 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1194/2392] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:06:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:00.206147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=25 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:10.149078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c3534-4950-441f-b694-472bd8ca7949 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:10.165224 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c3534-4950-441f-b694-472bd8ca7949 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:10.172989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be7c3a-2584-4d34-8d26-9a69a36b2c5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:10.174295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be7c3a-2584-4d34-8d26-9a69a36b2c5d service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:10.174672 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be7c3a-2584-4d34-8d26-9a69a36b2c5d service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:10.175080 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be7c3a-2584-4d34-8d26-9a69a36b2c5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:10.177061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-87be7c3a-2584-4d34-8d26-9a69a36b2c5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:10.190295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be7c3a-2584-4d34-8d26-9a69a36b2c5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:10.192223 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be7c3a-2584-4d34-8d26-9a69a36b2c5d service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:10.203598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1199/2393] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:20.198925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=6 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:20.971846 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6bdd2-b324-4178-beed-d6f905a7cc4c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:20.973290 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-90a6bdd2-b324-4178-beed-d6f905a7cc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:21.029722 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6bdd2-b324-4178-beed-d6f905a7cc4c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:21.033577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e9986-bc11-4d66-9e63-925dc770afc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:21.034279 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e9986-bc11-4d66-9e63-925dc770afc3 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:21.034559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e9986-bc11-4d66-9e63-925dc770afc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:21.034849 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e9986-bc11-4d66-9e63-925dc770afc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:21.036619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-884e9986-bc11-4d66-9e63-925dc770afc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:21.100030 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e9986-bc11-4d66-9e63-925dc770afc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:21.101477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e9986-bc11-4d66-9e63-925dc770afc3 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:21.111954 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1195/2394] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:24.782720 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be7c3a-2584-4d34-8d26-9a69a36b2c5d service nova] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:24.786153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-87be7c3a-2584-4d34-8d26-9a69a36b2c5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:24.865985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be7c3a-2584-4d34-8d26-9a69a36b2c5d service nova] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:24.878293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bc3df-863f-48cc-9786-6efb76f566d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:24.878293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bc3df-863f-48cc-9786-6efb76f566d6 service placement] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:24.878293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bc3df-863f-48cc-9786-6efb76f566d6 service placement] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:24.878293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bc3df-863f-48cc-9786-6efb76f566d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:24.878709 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4a1bc3df-863f-48cc-9786-6efb76f566d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:24.885994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bc3df-863f-48cc-9786-6efb76f566d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:24.888539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bc3df-863f-48cc-9786-6efb76f566d6 service placement] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:24.898545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1200/2395] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:24.920992 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e9986-bc11-4d66-9e63-925dc770afc3 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:24.922384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-884e9986-bc11-4d66-9e63-925dc770afc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:24.936785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e9986-bc11-4d66-9e63-925dc770afc3 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:24.946956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05875efd-ba5b-4bfc-ac4a-49efa032176d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:24.946956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05875efd-ba5b-4bfc-ac4a-49efa032176d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:24.946956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05875efd-ba5b-4bfc-ac4a-49efa032176d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:24.946956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05875efd-ba5b-4bfc-ac4a-49efa032176d admin admin] PATH_INFO: `/v3/users/dbee2da50ee6413db771071900703e29` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:24.947423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05875efd-ba5b-4bfc-ac4a-49efa032176d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbee2da50ee6413db771071900703e29)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:24.947423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05875efd-ba5b-4bfc-ac4a-49efa032176d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:25.201072 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-05875efd-ba5b-4bfc-ac4a-49efa032176d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbee2da50ee6413db771071900703e29'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:25.207729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-05875efd-ba5b-4bfc-ac4a-49efa032176d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbee2da50ee6413db771071900703e29'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:25.208381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-05875efd-ba5b-4bfc-ac4a-49efa032176d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbee2da50ee6413db771071900703e29'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:25.220156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-05875efd-ba5b-4bfc-ac4a-49efa032176d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbee2da50ee6413db771071900703e29'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:25.329731 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-05875efd-ba5b-4bfc-ac4a-49efa032176d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbee2da50ee6413db771071900703e29'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:25.336067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1196/2396] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:07:24 2022] DELETE /identity/v3/users/dbee2da50ee6413db771071900703e29 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:25.599956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bc3df-863f-48cc-9786-6efb76f566d6 service placement] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:25.611692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bc3df-863f-48cc-9786-6efb76f566d6 service placement] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:25.679363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5347e2f-f5a8-4305-8224-30e870b00f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:25.680586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5347e2f-f5a8-4305-8224-30e870b00f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:25.680871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5347e2f-f5a8-4305-8224-30e870b00f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:25.681150 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5347e2f-f5a8-4305-8224-30e870b00f68 admin admin] PATH_INFO: `/v3/projects/3b0036437f8d4f2aaa19e707199916e3` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:25.682472 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5347e2f-f5a8-4305-8224-30e870b00f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0036437f8d4f2aaa19e707199916e3)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:25.684475 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5347e2f-f5a8-4305-8224-30e870b00f68 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:25.795394 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d5347e2f-f5a8-4305-8224-30e870b00f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0036437f8d4f2aaa19e707199916e3', 'request_id': 'req-d5347e2f-f5a8-4305-8224-30e870b00f68'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:25.882180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d5347e2f-f5a8-4305-8224-30e870b00f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0036437f8d4f2aaa19e707199916e3', 'request_id': 'req-d5347e2f-f5a8-4305-8224-30e870b00f68'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:25.889038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d5347e2f-f5a8-4305-8224-30e870b00f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0036437f8d4f2aaa19e707199916e3', 'request_id': 'req-d5347e2f-f5a8-4305-8224-30e870b00f68'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:25.899742 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d5347e2f-f5a8-4305-8224-30e870b00f68 admin admin] The token cache is being invalidate because project 3b0036437f8d4f2aaa19e707199916e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:25.900007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d5347e2f-f5a8-4305-8224-30e870b00f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:25.900643 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d5347e2f-f5a8-4305-8224-30e870b00f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:25.902226 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1201/2397] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:07:25 2022] DELETE /identity/v3/projects/3b0036437f8d4f2aaa19e707199916e3 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:25.908915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2c898-192d-405c-be60-f5bb351e2c1c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:25.909315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2c898-192d-405c-be60-f5bb351e2c1c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:25.909617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2c898-192d-405c-be60-f5bb351e2c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:25.928962 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-43d2c898-192d-405c-be60-f5bb351e2c1c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:25.930189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-43d2c898-192d-405c-be60-f5bb351e2c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:25.946781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:25.988119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1197/2398] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:25.995973 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729ade0-c5a3-4e65-82ec-f810f27134e5 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:25.996262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729ade0-c5a3-4e65-82ec-f810f27134e5 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:25.996537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729ade0-c5a3-4e65-82ec-f810f27134e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:26.015234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-7729ade0-c5a3-4e65-82ec-f810f27134e5 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:26.016644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7729ade0-c5a3-4e65-82ec-f810f27134e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:26.024930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1202/2399] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:26.029928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2c898-192d-405c-be60-f5bb351e2c1c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:26.039623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2c898-192d-405c-be60-f5bb351e2c1c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:26.044978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a82cf1-62aa-442d-9233-1cba4356dbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:26.045832 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a82cf1-62aa-442d-9233-1cba4356dbeb admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:26.046118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a82cf1-62aa-442d-9233-1cba4356dbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:26.046394 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a82cf1-62aa-442d-9233-1cba4356dbeb admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:26.047153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a82cf1-62aa-442d-9233-1cba4356dbeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:26.047464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a82cf1-62aa-442d-9233-1cba4356dbeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:26.049367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a82cf1-62aa-442d-9233-1cba4356dbeb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:26.061601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1198/2400] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:07:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:26.070928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729ade0-c5a3-4e65-82ec-f810f27134e5 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:26.080622 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729ade0-c5a3-4e65-82ec-f810f27134e5 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:26.085407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3b026-54b6-4e2c-8489-359d00dd3860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:26.086091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3b026-54b6-4e2c-8489-359d00dd3860 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:26.086368 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3b026-54b6-4e2c-8489-359d00dd3860 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:26.086644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3b026-54b6-4e2c-8489-359d00dd3860 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:26.088122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3b026-54b6-4e2c-8489-359d00dd3860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:26.090166 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3b026-54b6-4e2c-8489-359d00dd3860 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:26.101120 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:26.274705 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1203/2401] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:07:26 2022] POST /identity/v3/projects => generated 382 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:26.278750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a82cf1-62aa-442d-9233-1cba4356dbeb admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:26.288909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a82cf1-62aa-442d-9233-1cba4356dbeb admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:26.345364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e81cd2-225c-41f6-b817-b7d492a0aca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:26.346040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e81cd2-225c-41f6-b817-b7d492a0aca9 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:26.346322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e81cd2-225c-41f6-b817-b7d492a0aca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:26.346596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e81cd2-225c-41f6-b817-b7d492a0aca9 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:26.347336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e81cd2-225c-41f6-b817-b7d492a0aca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:26.349251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e81cd2-225c-41f6-b817-b7d492a0aca9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:26.443859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1199/2402] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:07:26 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:26.448185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3b026-54b6-4e2c-8489-359d00dd3860 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:26.458993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3b026-54b6-4e2c-8489-359d00dd3860 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:26.463563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763f43d-4925-4f2b-afb1-34afde7e9131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:26.464312 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763f43d-4925-4f2b-afb1-34afde7e9131 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:26.464600 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763f43d-4925-4f2b-afb1-34afde7e9131 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:26.464873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763f43d-4925-4f2b-afb1-34afde7e9131 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:26.465630 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763f43d-4925-4f2b-afb1-34afde7e9131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:26.467519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763f43d-4925-4f2b-afb1-34afde7e9131 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:26.477512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1204/2403] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:26 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:26.482028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e81cd2-225c-41f6-b817-b7d492a0aca9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:26.492248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e81cd2-225c-41f6-b817-b7d492a0aca9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:26.496672 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bab170-8bbf-42e3-bfa3-74d3f499b281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:26.498070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bab170-8bbf-42e3-bfa3-74d3f499b281 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:26.498343 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bab170-8bbf-42e3-bfa3-74d3f499b281 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:26.498616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bab170-8bbf-42e3-bfa3-74d3f499b281 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:26.499393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bab170-8bbf-42e3-bfa3-74d3f499b281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:26.500967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bab170-8bbf-42e3-bfa3-74d3f499b281 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:26.512721 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1200/2404] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:26 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:26.516941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763f43d-4925-4f2b-afb1-34afde7e9131 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:26.527032 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763f43d-4925-4f2b-afb1-34afde7e9131 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:26.532122 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1b149-b820-482b-89f4-810945141169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:26.533005 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1b149-b820-482b-89f4-810945141169 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:26.533278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1b149-b820-482b-89f4-810945141169 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:26.533549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1b149-b820-482b-89f4-810945141169 admin admin] PATH_INFO: `/v3/projects/964a80f47d764d4c95326a2da02a8382/users/591a9c2f8ff14f27b260038e88dc13cc/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:26.549425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1b149-b820-482b-89f4-810945141169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=964a80f47d764d4c95326a2da02a8382, user_id=591a9c2f8ff14f27b260038e88dc13cc, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:26.552106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1b149-b820-482b-89f4-810945141169 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:26.684099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1205/2405] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:07:26 2022] PUT /identity/v3/projects/964a80f47d764d4c95326a2da02a8382/users/591a9c2f8ff14f27b260038e88dc13cc/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:26.694133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8eac19-1d2b-4b7f-bb4b-196f4437b72e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:26.694424 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8eac19-1d2b-4b7f-bb4b-196f4437b72e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:26.694703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8eac19-1d2b-4b7f-bb4b-196f4437b72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:26.720934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bd8eac19-1d2b-4b7f-bb4b-196f4437b72e None None] MFA Rules not processed for user `591a9c2f8ff14f27b260038e88dc13cc`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:26.722134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bd8eac19-1d2b-4b7f-bb4b-196f4437b72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:26.784877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1201/2406] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:26.789211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1b149-b820-482b-89f4-810945141169 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:26.849270 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1b149-b820-482b-89f4-810945141169 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:26.853848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f84ae9-6561-458d-a251-060774e59fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:26.854504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f84ae9-6561-458d-a251-060774e59fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:26.854782 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f84ae9-6561-458d-a251-060774e59fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:26.855084 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f84ae9-6561-458d-a251-060774e59fd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:26.855770 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f84ae9-6561-458d-a251-060774e59fd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:26.858317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f84ae9-6561-458d-a251-060774e59fd3 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:26.976517 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1206/2407] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:07:26 2022] POST /identity/v3/projects => generated 382 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:26.980617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8eac19-1d2b-4b7f-bb4b-196f4437b72e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:26.991244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8eac19-1d2b-4b7f-bb4b-196f4437b72e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:27.003727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:27.049410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441b396-052c-47f2-ad91-2a1c4601a6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:27.050004 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441b396-052c-47f2-ad91-2a1c4601a6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:27.050285 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441b396-052c-47f2-ad91-2a1c4601a6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:27.050583 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441b396-052c-47f2-ad91-2a1c4601a6b3 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:27.051294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441b396-052c-47f2-ad91-2a1c4601a6b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:27.053235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441b396-052c-47f2-ad91-2a1c4601a6b3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:27.180006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1202/2408] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:07:26 2022] POST /identity/v3/users => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:27.184018 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f84ae9-6561-458d-a251-060774e59fd3 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:27.194433 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f84ae9-6561-458d-a251-060774e59fd3 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:27.198878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a1147-c1b2-4770-a121-0316ffe51fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:27.199537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a1147-c1b2-4770-a121-0316ffe51fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:27.199820 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a1147-c1b2-4770-a121-0316ffe51fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:27.200590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a1147-c1b2-4770-a121-0316ffe51fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:27.201390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a1147-c1b2-4770-a121-0316ffe51fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:27.203028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a1147-c1b2-4770-a121-0316ffe51fd8 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:27.214584 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1207/2409] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:27 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:27.219336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441b396-052c-47f2-ad91-2a1c4601a6b3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:27.229315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441b396-052c-47f2-ad91-2a1c4601a6b3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:27.234126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452eed3-f5b9-4f87-a981-668d56f51d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:27.235036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452eed3-f5b9-4f87-a981-668d56f51d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:27.235312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452eed3-f5b9-4f87-a981-668d56f51d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:27.235580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452eed3-f5b9-4f87-a981-668d56f51d2b admin admin] PATH_INFO: `/v3/projects/0b2eae0bb2d340029ee1980e94af4215/users/e9f3359cc81a4636986dd745f7fb07b0/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:27.250098 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452eed3-f5b9-4f87-a981-668d56f51d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2eae0bb2d340029ee1980e94af4215, user_id=e9f3359cc81a4636986dd745f7fb07b0, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:27.252761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452eed3-f5b9-4f87-a981-668d56f51d2b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:27.380238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1203/2410] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:07:27 2022] PUT /identity/v3/projects/0b2eae0bb2d340029ee1980e94af4215/users/e9f3359cc81a4636986dd745f7fb07b0/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:27.384273 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a1147-c1b2-4770-a121-0316ffe51fd8 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:27.395000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a1147-c1b2-4770-a121-0316ffe51fd8 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:27.449182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9f5f8-4e45-4db9-9705-c6d05c5e9b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:27.449849 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9f5f8-4e45-4db9-9705-c6d05c5e9b95 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:27.450206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9f5f8-4e45-4db9-9705-c6d05c5e9b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:27.451355 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9f5f8-4e45-4db9-9705-c6d05c5e9b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:27.451879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9f5f8-4e45-4db9-9705-c6d05c5e9b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:27.453458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9f5f8-4e45-4db9-9705-c6d05c5e9b95 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:27.463423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1208/2411] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:27 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:27.467875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452eed3-f5b9-4f87-a981-668d56f51d2b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:27.477880 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452eed3-f5b9-4f87-a981-668d56f51d2b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:27.482503 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b01bd-0ad2-4385-b388-8b8e4950b8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:27.483395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b01bd-0ad2-4385-b388-8b8e4950b8b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:27.483675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b01bd-0ad2-4385-b388-8b8e4950b8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:27.483944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b01bd-0ad2-4385-b388-8b8e4950b8b0 admin admin] PATH_INFO: `/v3/projects/0b2eae0bb2d340029ee1980e94af4215/users/e9f3359cc81a4636986dd745f7fb07b0/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:27.485888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b01bd-0ad2-4385-b388-8b8e4950b8b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2eae0bb2d340029ee1980e94af4215, user_id=e9f3359cc81a4636986dd745f7fb07b0, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:27.488542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b01bd-0ad2-4385-b388-8b8e4950b8b0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:27.499093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1204/2412] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:07:27 2022] PUT /identity/v3/projects/0b2eae0bb2d340029ee1980e94af4215/users/e9f3359cc81a4636986dd745f7fb07b0/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:27.510234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1598b-f73c-4290-adfd-f63b04c723f3 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:27.510525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1598b-f73c-4290-adfd-f63b04c723f3 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:27.510806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1598b-f73c-4290-adfd-f63b04c723f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:27.537377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-90b1598b-f73c-4290-adfd-f63b04c723f3 None None] MFA Rules not processed for user `e9f3359cc81a4636986dd745f7fb07b0`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:27.538565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-90b1598b-f73c-4290-adfd-f63b04c723f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:27.543538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:27.606686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1209/2413] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:27.617500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b01bd-0ad2-4385-b388-8b8e4950b8b0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:27.619201 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a34b01bd-0ad2-4385-b388-8b8e4950b8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:27.677337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b01bd-0ad2-4385-b388-8b8e4950b8b0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:27.681356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8351e-af56-4178-8eb8-9a232eed5861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:27.682077 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8351e-af56-4178-8eb8-9a232eed5861 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:27.682392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8351e-af56-4178-8eb8-9a232eed5861 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:27.682711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8351e-af56-4178-8eb8-9a232eed5861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:27.692517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8351e-af56-4178-8eb8-9a232eed5861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:27.693934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8351e-af56-4178-8eb8-9a232eed5861 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:27.703202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1205/2414] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:27.801038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1598b-f73c-4290-adfd-f63b04c723f3 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:27.811942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1598b-f73c-4290-adfd-f63b04c723f3 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:27.815910 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17049c36-bc61-4c39-82f3-bbe2f44b4730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:27.816394 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17049c36-bc61-4c39-82f3-bbe2f44b4730 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:27.816640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17049c36-bc61-4c39-82f3-bbe2f44b4730 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:27.816640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17049c36-bc61-4c39-82f3-bbe2f44b4730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:27.873389 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17049c36-bc61-4c39-82f3-bbe2f44b4730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:27.874842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17049c36-bc61-4c39-82f3-bbe2f44b4730 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:27.884039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1210/2415] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:31.134751 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8351e-af56-4178-8eb8-9a232eed5861 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:31.198273 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8351e-af56-4178-8eb8-9a232eed5861 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:31.202838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:31.203720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:31.204002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:31.204277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] PATH_INFO: `/v3/users/591a9c2f8ff14f27b260038e88dc13cc` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:31.205497 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=591a9c2f8ff14f27b260038e88dc13cc)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:31.207530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:31.221777 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:31.479866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591a9c2f8ff14f27b260038e88dc13cc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:31.486859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591a9c2f8ff14f27b260038e88dc13cc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:31.487512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591a9c2f8ff14f27b260038e88dc13cc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:31.499996 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591a9c2f8ff14f27b260038e88dc13cc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:31.592272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591a9c2f8ff14f27b260038e88dc13cc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:31.599353 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1206/2416] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:07:31 2022] DELETE /identity/v3/users/591a9c2f8ff14f27b260038e88dc13cc => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:31.605168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692acec-850c-4ac4-82e5-dd026bfc9e3e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:31.605168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692acec-850c-4ac4-82e5-dd026bfc9e3e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:31.605168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692acec-850c-4ac4-82e5-dd026bfc9e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:31.622160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5692acec-850c-4ac4-82e5-dd026bfc9e3e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:31.623443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5692acec-850c-4ac4-82e5-dd026bfc9e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:31.684710 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1211/2417] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:31.694487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:31.695944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:31.753043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef58f1e-bfa4-4ce2-a526-cfd04ac27711 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:31.756725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470ab35-4590-4c75-a0c1-51289f7d67f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:31.757372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470ab35-4590-4c75-a0c1-51289f7d67f8 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:31.757656 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470ab35-4590-4c75-a0c1-51289f7d67f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:31.757931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470ab35-4590-4c75-a0c1-51289f7d67f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:31.767883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470ab35-4590-4c75-a0c1-51289f7d67f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:31.769364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470ab35-4590-4c75-a0c1-51289f7d67f8 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:31.778904 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1207/2418] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:32.205578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692acec-850c-4ac4-82e5-dd026bfc9e3e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:32.207105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5692acec-850c-4ac4-82e5-dd026bfc9e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:32.219378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692acec-850c-4ac4-82e5-dd026bfc9e3e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:32.225099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:32.226231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:32.231046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:32.231046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] PATH_INFO: `/v3/projects/964a80f47d764d4c95326a2da02a8382` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:32.231046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=964a80f47d764d4c95326a2da02a8382)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:32.231046 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:32.320684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:32.329045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '964a80f47d764d4c95326a2da02a8382', 'request_id': 'req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:32.420851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '964a80f47d764d4c95326a2da02a8382', 'request_id': 'req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:32.428477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '964a80f47d764d4c95326a2da02a8382', 'request_id': 'req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:32.438612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] The token cache is being invalidate because project 964a80f47d764d4c95326a2da02a8382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:32.438929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:32.440194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:32.441768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1212/2419] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:07:32 2022] DELETE /identity/v3/projects/964a80f47d764d4c95326a2da02a8382 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:32.445995 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470ab35-4590-4c75-a0c1-51289f7d67f8 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:32.456147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470ab35-4590-4c75-a0c1-51289f7d67f8 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:32.511779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd7840-3b84-4c76-8658-bd2828a15b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:32.512657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd7840-3b84-4c76-8658-bd2828a15b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:32.512934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd7840-3b84-4c76-8658-bd2828a15b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:32.513202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd7840-3b84-4c76-8658-bd2828a15b4e admin admin] PATH_INFO: `/v3/users/e9f3359cc81a4636986dd745f7fb07b0` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:32.514359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd7840-3b84-4c76-8658-bd2828a15b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f3359cc81a4636986dd745f7fb07b0)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:32.516299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd7840-3b84-4c76-8658-bd2828a15b4e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:32.716146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:32.724863 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d3bd7840-3b84-4c76-8658-bd2828a15b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f3359cc81a4636986dd745f7fb07b0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:32.731649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d3bd7840-3b84-4c76-8658-bd2828a15b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f3359cc81a4636986dd745f7fb07b0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:32.732335 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d3bd7840-3b84-4c76-8658-bd2828a15b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f3359cc81a4636986dd745f7fb07b0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:32.745053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d3bd7840-3b84-4c76-8658-bd2828a15b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f3359cc81a4636986dd745f7fb07b0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:32.826263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d3bd7840-3b84-4c76-8658-bd2828a15b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f3359cc81a4636986dd745f7fb07b0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:32.833115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1208/2420] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:07:32 2022] DELETE /identity/v3/users/e9f3359cc81a4636986dd745f7fb07b0 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:07:33.222442 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:33.223953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:33.237893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e7bcb-a59c-4b2b-b80d-bb0b580f3604 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:33.294508 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378851d4-2e39-4857-9967-12bb4b4ef26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:33.295380 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378851d4-2e39-4857-9967-12bb4b4ef26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:33.295664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378851d4-2e39-4857-9967-12bb4b4ef26a admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:33.295930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378851d4-2e39-4857-9967-12bb4b4ef26a admin admin] PATH_INFO: `/v3/projects/0b2eae0bb2d340029ee1980e94af4215` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:33.297083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378851d4-2e39-4857-9967-12bb4b4ef26a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2eae0bb2d340029ee1980e94af4215)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:33.299450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378851d4-2e39-4857-9967-12bb4b4ef26a admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:33.477837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-378851d4-2e39-4857-9967-12bb4b4ef26a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2eae0bb2d340029ee1980e94af4215', 'request_id': 'req-378851d4-2e39-4857-9967-12bb4b4ef26a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:33.580668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-378851d4-2e39-4857-9967-12bb4b4ef26a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2eae0bb2d340029ee1980e94af4215', 'request_id': 'req-378851d4-2e39-4857-9967-12bb4b4ef26a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:33.587637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-378851d4-2e39-4857-9967-12bb4b4ef26a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2eae0bb2d340029ee1980e94af4215', 'request_id': 'req-378851d4-2e39-4857-9967-12bb4b4ef26a'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:33.598561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-378851d4-2e39-4857-9967-12bb4b4ef26a admin admin] The token cache is being invalidate because project 0b2eae0bb2d340029ee1980e94af4215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:33.598877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-378851d4-2e39-4857-9967-12bb4b4ef26a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:33.599510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-378851d4-2e39-4857-9967-12bb4b4ef26a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:33.601157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1213/2421] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:07:33 2022] DELETE /identity/v3/projects/0b2eae0bb2d340029ee1980e94af4215 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:33.606426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fc910-cca7-4b98-8333-1f3bea9e07fa None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:33.606947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fc910-cca7-4b98-8333-1f3bea9e07fa None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:33.607053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fc910-cca7-4b98-8333-1f3bea9e07fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:33.625681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-044fc910-cca7-4b98-8333-1f3bea9e07fa None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:33.629977 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-044fc910-cca7-4b98-8333-1f3bea9e07fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:33.686693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1209/2422] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:33.695097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84a4ee-4c50-4ec9-af04-6e99e362872d None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:33.695381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84a4ee-4c50-4ec9-af04-6e99e362872d None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:33.695702 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84a4ee-4c50-4ec9-af04-6e99e362872d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:33.714005 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-fe84a4ee-4c50-4ec9-af04-6e99e362872d None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:33.715237 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-fe84a4ee-4c50-4ec9-af04-6e99e362872d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:33.723818 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1214/2423] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:07:33.728318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fc910-cca7-4b98-8333-1f3bea9e07fa None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:33.738422 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fc910-cca7-4b98-8333-1f3bea9e07fa None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:33.743491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498f7b3-41f2-471a-9292-313f3e0884c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:33.744156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498f7b3-41f2-471a-9292-313f3e0884c4 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:33.744436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498f7b3-41f2-471a-9292-313f3e0884c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:33.744709 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498f7b3-41f2-471a-9292-313f3e0884c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:33.745508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498f7b3-41f2-471a-9292-313f3e0884c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:33.745815 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498f7b3-41f2-471a-9292-313f3e0884c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:33.747452 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498f7b3-41f2-471a-9292-313f3e0884c4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:33.759811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1210/2424] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:07:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:33.764194 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84a4ee-4c50-4ec9-af04-6e99e362872d None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:33.773711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84a4ee-4c50-4ec9-af04-6e99e362872d None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:33.778315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df07d36-2c08-4c91-88c0-08076a36d6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:33.779003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df07d36-2c08-4c91-88c0-08076a36d6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:33.779441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df07d36-2c08-4c91-88c0-08076a36d6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:33.779724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df07d36-2c08-4c91-88c0-08076a36d6f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:33.780435 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df07d36-2c08-4c91-88c0-08076a36d6f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:33.782305 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df07d36-2c08-4c91-88c0-08076a36d6f3 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:33.856450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1215/2425] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:07:33 2022] POST /identity/v3/projects => generated 382 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:07:33.860460 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498f7b3-41f2-471a-9292-313f3e0884c4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:33.871202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498f7b3-41f2-471a-9292-313f3e0884c4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:33.928973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d144ae-3b67-4996-ab5f-841c0222b195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:33.929638 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d144ae-3b67-4996-ab5f-841c0222b195 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:33.929916 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d144ae-3b67-4996-ab5f-841c0222b195 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:33.930185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d144ae-3b67-4996-ab5f-841c0222b195 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:33.930895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d144ae-3b67-4996-ab5f-841c0222b195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:33.932802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d144ae-3b67-4996-ab5f-841c0222b195 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:34.004306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1211/2426] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:07:33 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:34.008482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df07d36-2c08-4c91-88c0-08076a36d6f3 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:34.018717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df07d36-2c08-4c91-88c0-08076a36d6f3 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:34.023251 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186020c8-d660-436b-8426-d6c0820ee681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:34.023908 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186020c8-d660-436b-8426-d6c0820ee681 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:34.024186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186020c8-d660-436b-8426-d6c0820ee681 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:34.024456 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186020c8-d660-436b-8426-d6c0820ee681 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:34.025168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186020c8-d660-436b-8426-d6c0820ee681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:34.026760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186020c8-d660-436b-8426-d6c0820ee681 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:34.037682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1216/2427] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:34 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:34.041773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d144ae-3b67-4996-ab5f-841c0222b195 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:34.052636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d144ae-3b67-4996-ab5f-841c0222b195 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:34.057139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66042ba7-fe41-42e4-b39a-35673326e62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:34.057794 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66042ba7-fe41-42e4-b39a-35673326e62c admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:34.058069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66042ba7-fe41-42e4-b39a-35673326e62c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:34.058337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66042ba7-fe41-42e4-b39a-35673326e62c admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:34.059113 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66042ba7-fe41-42e4-b39a-35673326e62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:34.060688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66042ba7-fe41-42e4-b39a-35673326e62c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:34.070636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1212/2428] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:34 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:34.074709 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186020c8-d660-436b-8426-d6c0820ee681 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:34.085418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186020c8-d660-436b-8426-d6c0820ee681 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:34.090235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409980e4-7a33-497f-b635-9973e1f2f21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:34.091208 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409980e4-7a33-497f-b635-9973e1f2f21e admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:34.091493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409980e4-7a33-497f-b635-9973e1f2f21e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:34.091771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409980e4-7a33-497f-b635-9973e1f2f21e admin admin] PATH_INFO: `/v3/projects/7bb452e1e54044a391f0b452c6b65b89/users/f556137115ce4f1f8bc9c047a1a00860/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:34.106171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:34.109213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409980e4-7a33-497f-b635-9973e1f2f21e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bb452e1e54044a391f0b452c6b65b89, user_id=f556137115ce4f1f8bc9c047a1a00860, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:34.112632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409980e4-7a33-497f-b635-9973e1f2f21e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:34.175796 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1217/2429] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:07:34 2022] PUT /identity/v3/projects/7bb452e1e54044a391f0b452c6b65b89/users/f556137115ce4f1f8bc9c047a1a00860/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:34.186020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cbefc-3548-4728-a4d1-c06df4aaec82 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:34.186314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cbefc-3548-4728-a4d1-c06df4aaec82 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:34.186598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cbefc-3548-4728-a4d1-c06df4aaec82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:34.213823 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f55cbefc-3548-4728-a4d1-c06df4aaec82 None None] MFA Rules not processed for user `f556137115ce4f1f8bc9c047a1a00860`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:34.215071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f55cbefc-3548-4728-a4d1-c06df4aaec82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:34.224240 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:34.280088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1213/2430] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:34.285187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409980e4-7a33-497f-b635-9973e1f2f21e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:34.348955 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409980e4-7a33-497f-b635-9973e1f2f21e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:34.353663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0bb88-f55d-413d-80fa-71392ff6d99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:34.354328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0bb88-f55d-413d-80fa-71392ff6d99c admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:34.354608 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0bb88-f55d-413d-80fa-71392ff6d99c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:34.354918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0bb88-f55d-413d-80fa-71392ff6d99c admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:34.355617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0bb88-f55d-413d-80fa-71392ff6d99c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:34.357506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0bb88-f55d-413d-80fa-71392ff6d99c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:34.588270 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1218/2431] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:07:34 2022] POST /identity/v3/projects => generated 380 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:34.592467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cbefc-3548-4728-a4d1-c06df4aaec82 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:34.604022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cbefc-3548-4728-a4d1-c06df4aaec82 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:34.661491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ed154-e8d9-4790-a861-ddf85527f9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:34.662177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ed154-e8d9-4790-a861-ddf85527f9cf admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:34.662363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ed154-e8d9-4790-a861-ddf85527f9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:34.662665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ed154-e8d9-4790-a861-ddf85527f9cf admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:34.663415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ed154-e8d9-4790-a861-ddf85527f9cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:34.665464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ed154-e8d9-4790-a861-ddf85527f9cf admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:34.840214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1214/2432] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:07:34 2022] POST /identity/v3/users => generated 297 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:34.844110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0bb88-f55d-413d-80fa-71392ff6d99c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:34.854578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0bb88-f55d-413d-80fa-71392ff6d99c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:34.859000 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52b839-bd2f-4c8c-bcf7-da9eee3160ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:34.859658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52b839-bd2f-4c8c-bcf7-da9eee3160ec admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:34.859935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52b839-bd2f-4c8c-bcf7-da9eee3160ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:34.860207 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52b839-bd2f-4c8c-bcf7-da9eee3160ec admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:34.860948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52b839-bd2f-4c8c-bcf7-da9eee3160ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:34.862529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52b839-bd2f-4c8c-bcf7-da9eee3160ec admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:34.873690 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1219/2433] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:34 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:34.880231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ed154-e8d9-4790-a861-ddf85527f9cf admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:34.890638 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ed154-e8d9-4790-a861-ddf85527f9cf admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:34.895890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee63cd-cd05-470a-b947-50c94834a641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:34.896768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee63cd-cd05-470a-b947-50c94834a641 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:34.897041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee63cd-cd05-470a-b947-50c94834a641 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:34.897306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee63cd-cd05-470a-b947-50c94834a641 admin admin] PATH_INFO: `/v3/projects/5d92d0240fc44d5a8effbcaf8177352a/users/d435c6e11ec64d728aac5cc6b8fe7268/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:34.921161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee63cd-cd05-470a-b947-50c94834a641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d92d0240fc44d5a8effbcaf8177352a, user_id=d435c6e11ec64d728aac5cc6b8fe7268, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:34.921161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee63cd-cd05-470a-b947-50c94834a641 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:35.016015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1215/2434] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:07:34 2022] PUT /identity/v3/projects/5d92d0240fc44d5a8effbcaf8177352a/users/d435c6e11ec64d728aac5cc6b8fe7268/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:35.020083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52b839-bd2f-4c8c-bcf7-da9eee3160ec admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:35.030301 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52b839-bd2f-4c8c-bcf7-da9eee3160ec admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:35.089524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958be49-7625-49bd-9082-dfcb4f42c738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:35.090197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958be49-7625-49bd-9082-dfcb4f42c738 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:35.090495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958be49-7625-49bd-9082-dfcb4f42c738 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:35.090750 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958be49-7625-49bd-9082-dfcb4f42c738 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:35.091510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958be49-7625-49bd-9082-dfcb4f42c738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:35.093106 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958be49-7625-49bd-9082-dfcb4f42c738 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:35.104037 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1220/2435] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:35 2022] GET /identity/v3/roles => generated 1427 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:35.108336 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee63cd-cd05-470a-b947-50c94834a641 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:35.119156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee63cd-cd05-470a-b947-50c94834a641 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:35.123714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08aa6fa-8fb2-491d-8f87-8675e8665a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:35.128850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08aa6fa-8fb2-491d-8f87-8675e8665a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:35.128850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08aa6fa-8fb2-491d-8f87-8675e8665a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:35.128850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08aa6fa-8fb2-491d-8f87-8675e8665a5a admin admin] PATH_INFO: `/v3/projects/5d92d0240fc44d5a8effbcaf8177352a/users/d435c6e11ec64d728aac5cc6b8fe7268/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:35.128850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08aa6fa-8fb2-491d-8f87-8675e8665a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d92d0240fc44d5a8effbcaf8177352a, user_id=d435c6e11ec64d728aac5cc6b8fe7268, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:35.129694 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08aa6fa-8fb2-491d-8f87-8675e8665a5a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:35.140500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1216/2436] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:07:35 2022] PUT /identity/v3/projects/5d92d0240fc44d5a8effbcaf8177352a/users/d435c6e11ec64d728aac5cc6b8fe7268/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:35.152188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e64ef5-ac1d-4297-9db5-bca2f282df44 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:35.152478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e64ef5-ac1d-4297-9db5-bca2f282df44 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:35.152755 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e64ef5-ac1d-4297-9db5-bca2f282df44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:35.180522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-87e64ef5-ac1d-4297-9db5-bca2f282df44 None None] MFA Rules not processed for user `d435c6e11ec64d728aac5cc6b8fe7268`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:35.181658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-87e64ef5-ac1d-4297-9db5-bca2f282df44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:35.228991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:35.250490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1221/2437] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:35.261973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08aa6fa-8fb2-491d-8f87-8675e8665a5a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:35.263680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b08aa6fa-8fb2-491d-8f87-8675e8665a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:35.323074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08aa6fa-8fb2-491d-8f87-8675e8665a5a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:35.326802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf582f-19f7-46ef-b00e-b535fd4eec69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:35.327539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf582f-19f7-46ef-b00e-b535fd4eec69 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:35.327828 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf582f-19f7-46ef-b00e-b535fd4eec69 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:35.328105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf582f-19f7-46ef-b00e-b535fd4eec69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:35.337537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf582f-19f7-46ef-b00e-b535fd4eec69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:35.339079 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf582f-19f7-46ef-b00e-b535fd4eec69 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:35.348759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1217/2438] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:35.941440 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e64ef5-ac1d-4297-9db5-bca2f282df44 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:36.003415 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e64ef5-ac1d-4297-9db5-bca2f282df44 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:36.008569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:36.009442 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:36.009752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:36.010026 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] PATH_INFO: `/v3/users/f556137115ce4f1f8bc9c047a1a00860` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:36.011276 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f556137115ce4f1f8bc9c047a1a00860)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:36.013278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:36.252646 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f556137115ce4f1f8bc9c047a1a00860'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:36.260009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f556137115ce4f1f8bc9c047a1a00860'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:36.260683 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f556137115ce4f1f8bc9c047a1a00860'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:36.272722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f556137115ce4f1f8bc9c047a1a00860'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:36.403751 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f556137115ce4f1f8bc9c047a1a00860'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:36.410428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1222/2439] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:07:35 2022] DELETE /identity/v3/users/f556137115ce4f1f8bc9c047a1a00860 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:36.415647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd44901-e3ea-412a-b0f8-6c49787ca462 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:36.416127 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd44901-e3ea-412a-b0f8-6c49787ca462 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:36.416259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd44901-e3ea-412a-b0f8-6c49787ca462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:36.422894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:36.435151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8bd44901-e3ea-412a-b0f8-6c49787ca462 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:36.436383 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8bd44901-e3ea-412a-b0f8-6c49787ca462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:36.494400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1218/2440] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:36.503989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:36.505414 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:36.563365 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc177e7b-7a26-4f37-8976-c75ba929dc07 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:36.567213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0474dc-ea13-4240-8428-f158d7b05586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:36.567869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0474dc-ea13-4240-8428-f158d7b05586 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:36.568148 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0474dc-ea13-4240-8428-f158d7b05586 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:36.568425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0474dc-ea13-4240-8428-f158d7b05586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:36.577857 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0474dc-ea13-4240-8428-f158d7b05586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:36.579313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0474dc-ea13-4240-8428-f158d7b05586 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:36.589680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1223/2441] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:37.051338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd44901-e3ea-412a-b0f8-6c49787ca462 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:37.052741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8bd44901-e3ea-412a-b0f8-6c49787ca462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:37.064619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd44901-e3ea-412a-b0f8-6c49787ca462 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:37.069806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:37.070726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:37.071032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:37.071307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] PATH_INFO: `/v3/projects/7bb452e1e54044a391f0b452c6b65b89` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:37.072504 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bb452e1e54044a391f0b452c6b65b89)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:37.074423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:37.234967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bb452e1e54044a391f0b452c6b65b89', 'request_id': 'req-d1d61172-046d-4452-a38c-ebd871e6d9a0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:37.329349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bb452e1e54044a391f0b452c6b65b89', 'request_id': 'req-d1d61172-046d-4452-a38c-ebd871e6d9a0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:37.336859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bb452e1e54044a391f0b452c6b65b89', 'request_id': 'req-d1d61172-046d-4452-a38c-ebd871e6d9a0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:37.349916 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] The token cache is being invalidate because project 7bb452e1e54044a391f0b452c6b65b89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:37.350237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:37.350902 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:37.352461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1219/2442] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:07:37 2022] DELETE /identity/v3/projects/7bb452e1e54044a391f0b452c6b65b89 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:37.356918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0474dc-ea13-4240-8428-f158d7b05586 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:37.367642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0474dc-ea13-4240-8428-f158d7b05586 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:37.372299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:37.425928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4a370-8b06-4d2d-b7f7-f7ff6c388e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:37.426805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4a370-8b06-4d2d-b7f7-f7ff6c388e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:37.427118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4a370-8b06-4d2d-b7f7-f7ff6c388e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:37.427396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4a370-8b06-4d2d-b7f7-f7ff6c388e7d admin admin] PATH_INFO: `/v3/users/d435c6e11ec64d728aac5cc6b8fe7268` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:37.429293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4a370-8b06-4d2d-b7f7-f7ff6c388e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d435c6e11ec64d728aac5cc6b8fe7268)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:37.431381 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4a370-8b06-4d2d-b7f7-f7ff6c388e7d admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:37.612090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bff4a370-8b06-4d2d-b7f7-f7ff6c388e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd435c6e11ec64d728aac5cc6b8fe7268'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:37.619656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bff4a370-8b06-4d2d-b7f7-f7ff6c388e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd435c6e11ec64d728aac5cc6b8fe7268'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:37.620361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bff4a370-8b06-4d2d-b7f7-f7ff6c388e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd435c6e11ec64d728aac5cc6b8fe7268'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:37.635780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bff4a370-8b06-4d2d-b7f7-f7ff6c388e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd435c6e11ec64d728aac5cc6b8fe7268'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:37.712438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bff4a370-8b06-4d2d-b7f7-f7ff6c388e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd435c6e11ec64d728aac5cc6b8fe7268'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:37.718716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1224/2443] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:07:37 2022] DELETE /identity/v3/users/d435c6e11ec64d728aac5cc6b8fe7268 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:07:38.189612 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:38.191177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:38.213175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d61172-046d-4452-a38c-ebd871e6d9a0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:38.276493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40328753-4e07-4f41-83a4-cbbe5783a427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:38.277496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40328753-4e07-4f41-83a4-cbbe5783a427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:38.277771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40328753-4e07-4f41-83a4-cbbe5783a427 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:38.278047 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40328753-4e07-4f41-83a4-cbbe5783a427 admin admin] PATH_INFO: `/v3/projects/5d92d0240fc44d5a8effbcaf8177352a` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:38.279312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40328753-4e07-4f41-83a4-cbbe5783a427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d92d0240fc44d5a8effbcaf8177352a)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:38.281224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40328753-4e07-4f41-83a4-cbbe5783a427 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:38.409570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:38.422265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-40328753-4e07-4f41-83a4-cbbe5783a427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d92d0240fc44d5a8effbcaf8177352a', 'request_id': 'req-40328753-4e07-4f41-83a4-cbbe5783a427'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:38.598470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-40328753-4e07-4f41-83a4-cbbe5783a427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d92d0240fc44d5a8effbcaf8177352a', 'request_id': 'req-40328753-4e07-4f41-83a4-cbbe5783a427'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:38.605803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-40328753-4e07-4f41-83a4-cbbe5783a427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d92d0240fc44d5a8effbcaf8177352a', 'request_id': 'req-40328753-4e07-4f41-83a4-cbbe5783a427'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:38.616825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-40328753-4e07-4f41-83a4-cbbe5783a427 admin admin] The token cache is being invalidate because project 5d92d0240fc44d5a8effbcaf8177352a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:38.617148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-40328753-4e07-4f41-83a4-cbbe5783a427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:38.618016 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-40328753-4e07-4f41-83a4-cbbe5783a427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:38.619539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1220/2444] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:07:38 2022] DELETE /identity/v3/projects/5d92d0240fc44d5a8effbcaf8177352a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:38.624441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1947ff-3562-4bea-b373-de3085363e02 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:38.624923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1947ff-3562-4bea-b373-de3085363e02 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:38.625078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1947ff-3562-4bea-b373-de3085363e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:38.644314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-ff1947ff-3562-4bea-b373-de3085363e02 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:38.645856 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ff1947ff-3562-4bea-b373-de3085363e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:38.714484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1225/2445] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:38.722351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768851c2-50c3-4208-acc8-da8dfdd844ea None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:38.722637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768851c2-50c3-4208-acc8-da8dfdd844ea None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:38.722954 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768851c2-50c3-4208-acc8-da8dfdd844ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:38.744335 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-768851c2-50c3-4208-acc8-da8dfdd844ea None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:38.745565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-768851c2-50c3-4208-acc8-da8dfdd844ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:38.753930 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1221/2446] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:38.759353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1947ff-3562-4bea-b373-de3085363e02 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:38.769216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1947ff-3562-4bea-b373-de3085363e02 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:38.774311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbeaceb-e711-4542-a142-df4dd8f3bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:38.775025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbeaceb-e711-4542-a142-df4dd8f3bd39 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:38.775308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbeaceb-e711-4542-a142-df4dd8f3bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:38.775582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbeaceb-e711-4542-a142-df4dd8f3bd39 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:38.776311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbeaceb-e711-4542-a142-df4dd8f3bd39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:38.776619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbeaceb-e711-4542-a142-df4dd8f3bd39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:38.778307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbeaceb-e711-4542-a142-df4dd8f3bd39 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:38.791156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1226/2447] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:07:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:38.796217 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768851c2-50c3-4208-acc8-da8dfdd844ea None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:38.806532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768851c2-50c3-4208-acc8-da8dfdd844ea None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:38.811409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e8faa-8b35-493a-8581-abdf7a45a565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:38.812066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e8faa-8b35-493a-8581-abdf7a45a565 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:38.812349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e8faa-8b35-493a-8581-abdf7a45a565 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:38.812616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e8faa-8b35-493a-8581-abdf7a45a565 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:38.813322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e8faa-8b35-493a-8581-abdf7a45a565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:38.815259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e8faa-8b35-493a-8581-abdf7a45a565 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:38.913262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1222/2448] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:07:38 2022] POST /identity/v3/projects => generated 386 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:38.917401 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbeaceb-e711-4542-a142-df4dd8f3bd39 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:38.927666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbeaceb-e711-4542-a142-df4dd8f3bd39 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:38.939582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:38.983860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f830a-bcc9-48a2-8dd0-a982a2d485b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:38.984542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f830a-bcc9-48a2-8dd0-a982a2d485b6 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:38.984823 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f830a-bcc9-48a2-8dd0-a982a2d485b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:38.985098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f830a-bcc9-48a2-8dd0-a982a2d485b6 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:38.985784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f830a-bcc9-48a2-8dd0-a982a2d485b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:38.987728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f830a-bcc9-48a2-8dd0-a982a2d485b6 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:39.102411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1227/2449] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:07:38 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:39.107305 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e8faa-8b35-493a-8581-abdf7a45a565 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:39.118771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e8faa-8b35-493a-8581-abdf7a45a565 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:39.123359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7587fa7-5601-4d2f-93d1-f991a3ace9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:39.124020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7587fa7-5601-4d2f-93d1-f991a3ace9fb admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:39.124300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7587fa7-5601-4d2f-93d1-f991a3ace9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:39.124578 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7587fa7-5601-4d2f-93d1-f991a3ace9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:39.125297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7587fa7-5601-4d2f-93d1-f991a3ace9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:39.126951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7587fa7-5601-4d2f-93d1-f991a3ace9fb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:39.138027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1223/2450] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:39 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:07:39.142211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f830a-bcc9-48a2-8dd0-a982a2d485b6 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:39.153061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f830a-bcc9-48a2-8dd0-a982a2d485b6 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:39.157725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fdeda-3711-43a5-be3e-56e08efdd6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:39.158391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fdeda-3711-43a5-be3e-56e08efdd6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:39.158674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fdeda-3711-43a5-be3e-56e08efdd6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:39.158994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fdeda-3711-43a5-be3e-56e08efdd6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:39.159716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051fdeda-3711-43a5-be3e-56e08efdd6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:39.161407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051fdeda-3711-43a5-be3e-56e08efdd6b8 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:39.177482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1228/2451] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:39 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:39.181579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7587fa7-5601-4d2f-93d1-f991a3ace9fb admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:39.191907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7587fa7-5601-4d2f-93d1-f991a3ace9fb admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:39.196708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698d982-c86b-4f6f-8c2c-08d20ba743ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:39.197816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698d982-c86b-4f6f-8c2c-08d20ba743ea admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:39.198001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698d982-c86b-4f6f-8c2c-08d20ba743ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:39.198302 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698d982-c86b-4f6f-8c2c-08d20ba743ea admin admin] PATH_INFO: `/v3/projects/2568c44d338a46e1ad716690b0a32487/users/f69e35cf5bcd43619fd0f7e303969d7d/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:39.214280 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698d982-c86b-4f6f-8c2c-08d20ba743ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2568c44d338a46e1ad716690b0a32487, user_id=f69e35cf5bcd43619fd0f7e303969d7d, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:39.216988 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698d982-c86b-4f6f-8c2c-08d20ba743ea admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:39.358778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1224/2452] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:07:39 2022] PUT /identity/v3/projects/2568c44d338a46e1ad716690b0a32487/users/f69e35cf5bcd43619fd0f7e303969d7d/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:07:39.370931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126dc78-0601-4f99-9c85-b9a9b5e0e2a5 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:39.371299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126dc78-0601-4f99-9c85-b9a9b5e0e2a5 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:39.371477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126dc78-0601-4f99-9c85-b9a9b5e0e2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:39.399601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-e126dc78-0601-4f99-9c85-b9a9b5e0e2a5 None None] MFA Rules not processed for user `f69e35cf5bcd43619fd0f7e303969d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:39.401250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e126dc78-0601-4f99-9c85-b9a9b5e0e2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:39.471070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1229/2453] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:39.483070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698d982-c86b-4f6f-8c2c-08d20ba743ea admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:39.484780 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b698d982-c86b-4f6f-8c2c-08d20ba743ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:39.545306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698d982-c86b-4f6f-8c2c-08d20ba743ea admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:39.549324 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1658c-3ba2-4a72-a2ff-d312c2edb8d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:39.549983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1658c-3ba2-4a72-a2ff-d312c2edb8d7 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:39.550265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1658c-3ba2-4a72-a2ff-d312c2edb8d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:39.550574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1658c-3ba2-4a72-a2ff-d312c2edb8d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:39.561492 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1658c-3ba2-4a72-a2ff-d312c2edb8d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:39.562991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1658c-3ba2-4a72-a2ff-d312c2edb8d7 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:39.572977 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1225/2454] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:39.592993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126dc78-0601-4f99-9c85-b9a9b5e0e2a5 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:39.656256 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126dc78-0601-4f99-9c85-b9a9b5e0e2a5 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:39.660989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:39.661922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:39.662206 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:39.662477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] PATH_INFO: `/v3/users/f69e35cf5bcd43619fd0f7e303969d7d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:39.664360 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69e35cf5bcd43619fd0f7e303969d7d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:39.666678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:39.764769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:39.882156 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69e35cf5bcd43619fd0f7e303969d7d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:39.889982 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69e35cf5bcd43619fd0f7e303969d7d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:39.890737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69e35cf5bcd43619fd0f7e303969d7d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:39.904897 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69e35cf5bcd43619fd0f7e303969d7d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:40.003893 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69e35cf5bcd43619fd0f7e303969d7d'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:40.011072 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1230/2455] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:07:39 2022] DELETE /identity/v3/users/f69e35cf5bcd43619fd0f7e303969d7d => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:40.015950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7620780-ff99-4976-ad89-95863882ea7b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:40.016170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7620780-ff99-4976-ad89-95863882ea7b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:40.016445 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7620780-ff99-4976-ad89-95863882ea7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:40.037508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e7620780-ff99-4976-ad89-95863882ea7b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:40.038749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e7620780-ff99-4976-ad89-95863882ea7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:40.091727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:40.105469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1226/2456] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:40.116403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:40.118120 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:40.177612 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e3eb1-fa64-40db-b887-5e6175c01ad9 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:40.182019 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f41e5b-802a-4d0a-b23c-efed88074af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:40.182765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f41e5b-802a-4d0a-b23c-efed88074af3 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:40.183123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f41e5b-802a-4d0a-b23c-efed88074af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:40.183402 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f41e5b-802a-4d0a-b23c-efed88074af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:40.194024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f41e5b-802a-4d0a-b23c-efed88074af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:40.196190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f41e5b-802a-4d0a-b23c-efed88074af3 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:40.210013 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1231/2457] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:40.760264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7620780-ff99-4976-ad89-95863882ea7b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:40.761481 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e7620780-ff99-4976-ad89-95863882ea7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:40.774356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7620780-ff99-4976-ad89-95863882ea7b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:40.779392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89663197-bf5e-493d-947f-7f13761b19dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:40.780476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89663197-bf5e-493d-947f-7f13761b19dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:40.780617 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89663197-bf5e-493d-947f-7f13761b19dc admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:40.780810 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89663197-bf5e-493d-947f-7f13761b19dc admin admin] PATH_INFO: `/v3/projects/2568c44d338a46e1ad716690b0a32487` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:40.782006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89663197-bf5e-493d-947f-7f13761b19dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2568c44d338a46e1ad716690b0a32487)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:40.783962 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89663197-bf5e-493d-947f-7f13761b19dc admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:40.924839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-89663197-bf5e-493d-947f-7f13761b19dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2568c44d338a46e1ad716690b0a32487', 'request_id': 'req-89663197-bf5e-493d-947f-7f13761b19dc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:41.037064 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-89663197-bf5e-493d-947f-7f13761b19dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2568c44d338a46e1ad716690b0a32487', 'request_id': 'req-89663197-bf5e-493d-947f-7f13761b19dc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:41.044746 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-89663197-bf5e-493d-947f-7f13761b19dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2568c44d338a46e1ad716690b0a32487', 'request_id': 'req-89663197-bf5e-493d-947f-7f13761b19dc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:41.058262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-89663197-bf5e-493d-947f-7f13761b19dc admin admin] The token cache is being invalidate because project 2568c44d338a46e1ad716690b0a32487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:41.058553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-89663197-bf5e-493d-947f-7f13761b19dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:41.059284 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-89663197-bf5e-493d-947f-7f13761b19dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:41.060818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1227/2458] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:07:40 2022] DELETE /identity/v3/projects/2568c44d338a46e1ad716690b0a32487 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:41.065651 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90190c8-6d4d-4451-9f3d-d64fa3464a8b None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:41.066132 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90190c8-6d4d-4451-9f3d-d64fa3464a8b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:41.066263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90190c8-6d4d-4451-9f3d-d64fa3464a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:41.084176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c90190c8-6d4d-4451-9f3d-d64fa3464a8b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:41.086565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c90190c8-6d4d-4451-9f3d-d64fa3464a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:41.149237 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1232/2459] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:41.156946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1f6c6-1209-4419-a74e-70b25007db52 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:41.157238 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1f6c6-1209-4419-a74e-70b25007db52 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:41.157518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1f6c6-1209-4419-a74e-70b25007db52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:41.177292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d4e1f6c6-1209-4419-a74e-70b25007db52 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:41.178589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d4e1f6c6-1209-4419-a74e-70b25007db52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:41.187046 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1228/2460] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:41.191304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90190c8-6d4d-4451-9f3d-d64fa3464a8b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:41.201382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90190c8-6d4d-4451-9f3d-d64fa3464a8b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:41.205835 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235d407-8b28-4fdf-91b2-f6a6b3d11c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:41.206504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d407-8b28-4fdf-91b2-f6a6b3d11c45 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:41.206785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d407-8b28-4fdf-91b2-f6a6b3d11c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:41.207126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d407-8b28-4fdf-91b2-f6a6b3d11c45 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:41.207837 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235d407-8b28-4fdf-91b2-f6a6b3d11c45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:41.208138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235d407-8b28-4fdf-91b2-f6a6b3d11c45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:41.210256 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235d407-8b28-4fdf-91b2-f6a6b3d11c45 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:41.221970 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1233/2461] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:07:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:07:41.226877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1f6c6-1209-4419-a74e-70b25007db52 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:41.236158 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1f6c6-1209-4419-a74e-70b25007db52 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:41.240928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e5e01-fce3-4aa9-b6ed-9bd09093908f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:41.241489 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e5e01-fce3-4aa9-b6ed-9bd09093908f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:41.241769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e5e01-fce3-4aa9-b6ed-9bd09093908f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:41.242046 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e5e01-fce3-4aa9-b6ed-9bd09093908f admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:41.242724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e5e01-fce3-4aa9-b6ed-9bd09093908f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:41.244699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e5e01-fce3-4aa9-b6ed-9bd09093908f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:41.383837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1229/2462] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:07:41 2022] POST /identity/v3/projects => generated 384 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:07:41.388050 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235d407-8b28-4fdf-91b2-f6a6b3d11c45 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:41.398986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235d407-8b28-4fdf-91b2-f6a6b3d11c45 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:41.428754 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:41.454965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c89fb-b9ca-465b-8ada-62c96a9e94ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:41.455635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c89fb-b9ca-465b-8ada-62c96a9e94ca admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:41.455916 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c89fb-b9ca-465b-8ada-62c96a9e94ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:41.456190 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c89fb-b9ca-465b-8ada-62c96a9e94ca admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:41.456879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c89fb-b9ca-465b-8ada-62c96a9e94ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:41.458904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c89fb-b9ca-465b-8ada-62c96a9e94ca admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:41.604044 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1234/2463] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:07:41 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:41.608207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e5e01-fce3-4aa9-b6ed-9bd09093908f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:41.618465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e5e01-fce3-4aa9-b6ed-9bd09093908f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:41.623130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bda00c-e366-4feb-ba08-264921dec77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:41.623792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bda00c-e366-4feb-ba08-264921dec77a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:41.624074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bda00c-e366-4feb-ba08-264921dec77a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:41.624346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bda00c-e366-4feb-ba08-264921dec77a admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:41.625101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bda00c-e366-4feb-ba08-264921dec77a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:41.626726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bda00c-e366-4feb-ba08-264921dec77a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:41.637808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1230/2464] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:41 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:41.641831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c89fb-b9ca-465b-8ada-62c96a9e94ca admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:41.652713 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c89fb-b9ca-465b-8ada-62c96a9e94ca admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:41.657258 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ee2bd-6c8b-4325-af73-a9e4ebc9bfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:41.657927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ee2bd-6c8b-4325-af73-a9e4ebc9bfc4 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:41.658195 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ee2bd-6c8b-4325-af73-a9e4ebc9bfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:41.658465 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ee2bd-6c8b-4325-af73-a9e4ebc9bfc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:41.659220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ee2bd-6c8b-4325-af73-a9e4ebc9bfc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:41.660800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ee2bd-6c8b-4325-af73-a9e4ebc9bfc4 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:41.670943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1235/2465] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:41 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:41.675344 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bda00c-e366-4feb-ba08-264921dec77a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:41.686173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bda00c-e366-4feb-ba08-264921dec77a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:41.690786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ef3d0-8d45-4eca-94e9-f3b9700e0304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:41.691827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ef3d0-8d45-4eca-94e9-f3b9700e0304 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:41.692119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ef3d0-8d45-4eca-94e9-f3b9700e0304 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:41.692422 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ef3d0-8d45-4eca-94e9-f3b9700e0304 admin admin] PATH_INFO: `/v3/projects/b01e92dde7fc49b79f4b7c9d3444a4a8/users/7d6a3659c7b34c9db600893370b99205/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:41.708046 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ef3d0-8d45-4eca-94e9-f3b9700e0304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01e92dde7fc49b79f4b7c9d3444a4a8, user_id=7d6a3659c7b34c9db600893370b99205, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:41.710979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ef3d0-8d45-4eca-94e9-f3b9700e0304 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:41.805234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1231/2466] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:07:41 2022] PUT /identity/v3/projects/b01e92dde7fc49b79f4b7c9d3444a4a8/users/7d6a3659c7b34c9db600893370b99205/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:41.815784 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c8a13-2419-4613-ac92-4494767ccfcc None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:41.816070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c8a13-2419-4613-ac92-4494767ccfcc None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:41.816345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c8a13-2419-4613-ac92-4494767ccfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:41.843688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d40c8a13-2419-4613-ac92-4494767ccfcc None None] MFA Rules not processed for user `7d6a3659c7b34c9db600893370b99205`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:41.845149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d40c8a13-2419-4613-ac92-4494767ccfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:41.911372 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1236/2467] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:41.915889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ef3d0-8d45-4eca-94e9-f3b9700e0304 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:41.977705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ef3d0-8d45-4eca-94e9-f3b9700e0304 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:41.982326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345795a7-8ff3-4835-b509-f518160a8fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:41.983021 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345795a7-8ff3-4835-b509-f518160a8fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:41.983308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345795a7-8ff3-4835-b509-f518160a8fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:41.983614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345795a7-8ff3-4835-b509-f518160a8fc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:41.984308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345795a7-8ff3-4835-b509-f518160a8fc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:41.986219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345795a7-8ff3-4835-b509-f518160a8fc9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:42.074373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1232/2468] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:07:41 2022] POST /identity/v3/projects => generated 384 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:42.078485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c8a13-2419-4613-ac92-4494767ccfcc None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:42.090136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c8a13-2419-4613-ac92-4494767ccfcc None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:42.150536 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccc69c-73fc-4596-95ce-c8deda99bf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:42.151261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccc69c-73fc-4596-95ce-c8deda99bf7b admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.151547 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccc69c-73fc-4596-95ce-c8deda99bf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.151822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccc69c-73fc-4596-95ce-c8deda99bf7b admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.152532 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccc69c-73fc-4596-95ce-c8deda99bf7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:42.154481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccc69c-73fc-4596-95ce-c8deda99bf7b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:42.273443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1237/2469] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:07:42 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:42.277735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345795a7-8ff3-4835-b509-f518160a8fc9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:42.283146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:42.288971 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345795a7-8ff3-4835-b509-f518160a8fc9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:42.294611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bae643-f2c3-4592-9ccb-f4556d1b2dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:42.295360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bae643-f2c3-4592-9ccb-f4556d1b2dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.300388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bae643-f2c3-4592-9ccb-f4556d1b2dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.300388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bae643-f2c3-4592-9ccb-f4556d1b2dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.300388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bae643-f2c3-4592-9ccb-f4556d1b2dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:42.300388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bae643-f2c3-4592-9ccb-f4556d1b2dc5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:42.311166 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1233/2470] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:42 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:42.316361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccc69c-73fc-4596-95ce-c8deda99bf7b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:42.326463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccc69c-73fc-4596-95ce-c8deda99bf7b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:42.331199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794f48f-fdd1-4e04-9cfb-0cc52ad4ce61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:42.331870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794f48f-fdd1-4e04-9cfb-0cc52ad4ce61 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.332138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794f48f-fdd1-4e04-9cfb-0cc52ad4ce61 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.332410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794f48f-fdd1-4e04-9cfb-0cc52ad4ce61 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.333124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794f48f-fdd1-4e04-9cfb-0cc52ad4ce61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:42.334753 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794f48f-fdd1-4e04-9cfb-0cc52ad4ce61 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:42.345037 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1238/2471] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:42 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:42.349202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bae643-f2c3-4592-9ccb-f4556d1b2dc5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:42.360187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bae643-f2c3-4592-9ccb-f4556d1b2dc5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:42.365036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d891ebc-5ae9-4bba-b339-b6f9c4d28e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:42.366628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d891ebc-5ae9-4bba-b339-b6f9c4d28e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.366945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d891ebc-5ae9-4bba-b339-b6f9c4d28e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.367222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d891ebc-5ae9-4bba-b339-b6f9c4d28e34 admin admin] PATH_INFO: `/v3/projects/3aca008ae52b4c32bd07ed19da16d37e/users/f60d742e11934dddb285a4750f746191/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.382602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d891ebc-5ae9-4bba-b339-b6f9c4d28e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aca008ae52b4c32bd07ed19da16d37e, user_id=f60d742e11934dddb285a4750f746191, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:42.385339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d891ebc-5ae9-4bba-b339-b6f9c4d28e34 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:42.501282 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1234/2472] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:07:42 2022] PUT /identity/v3/projects/3aca008ae52b4c32bd07ed19da16d37e/users/f60d742e11934dddb285a4750f746191/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:42.513107 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37e6c0-e063-4900-89b5-5fc3483bbeec None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.513419 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37e6c0-e063-4900-89b5-5fc3483bbeec None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.513702 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37e6c0-e063-4900-89b5-5fc3483bbeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.544191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1f37e6c0-e063-4900-89b5-5fc3483bbeec None None] MFA Rules not processed for user `f60d742e11934dddb285a4750f746191`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:42.544191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1f37e6c0-e063-4900-89b5-5fc3483bbeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:42.552348 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:42.610284 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1239/2473] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:42.614984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d891ebc-5ae9-4bba-b339-b6f9c4d28e34 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:42.679379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d891ebc-5ae9-4bba-b339-b6f9c4d28e34 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:42.684484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4124a2-990d-4828-a241-590ccf1281c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:42.685169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4124a2-990d-4828-a241-590ccf1281c0 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.685445 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4124a2-990d-4828-a241-590ccf1281c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.685714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4124a2-990d-4828-a241-590ccf1281c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.686393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4124a2-990d-4828-a241-590ccf1281c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:42.688322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4124a2-990d-4828-a241-590ccf1281c0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:42.777573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1235/2474] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:07:42 2022] POST /identity/v3/projects => generated 386 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:42.781669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37e6c0-e063-4900-89b5-5fc3483bbeec None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:42.796362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37e6c0-e063-4900-89b5-5fc3483bbeec None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:42.854528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc91b7-770f-4a30-ac7c-0b64a18931e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:42.856598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc91b7-770f-4a30-ac7c-0b64a18931e4 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.856598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc91b7-770f-4a30-ac7c-0b64a18931e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.856598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc91b7-770f-4a30-ac7c-0b64a18931e4 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.856598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc91b7-770f-4a30-ac7c-0b64a18931e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:42.858396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc91b7-770f-4a30-ac7c-0b64a18931e4 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:42.958380 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1240/2475] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:07:42 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:42.963002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4124a2-990d-4828-a241-590ccf1281c0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:42.973380 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4124a2-990d-4828-a241-590ccf1281c0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:42.978601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cab61-dbd1-4b61-a2c5-6eb0e9f36d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:42.979319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cab61-dbd1-4b61-a2c5-6eb0e9f36d08 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.980715 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cab61-dbd1-4b61-a2c5-6eb0e9f36d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.980715 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cab61-dbd1-4b61-a2c5-6eb0e9f36d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.980715 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cab61-dbd1-4b61-a2c5-6eb0e9f36d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:42.982227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cab61-dbd1-4b61-a2c5-6eb0e9f36d08 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:43.002514 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1236/2476] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:42 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:07:43.007025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc91b7-770f-4a30-ac7c-0b64a18931e4 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:43.017343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc91b7-770f-4a30-ac7c-0b64a18931e4 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:43.023098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f513aec-886b-40dc-9e66-0d10a2c8d6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:43.024005 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f513aec-886b-40dc-9e66-0d10a2c8d6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:43.024438 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f513aec-886b-40dc-9e66-0d10a2c8d6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:43.025157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f513aec-886b-40dc-9e66-0d10a2c8d6d5 admin admin] PATH_INFO: `/v3/projects/743ade8649c241b4b43a0af5c9b96365/users/7388bc9cb6b64571bd25c3b6f6c2172b/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:43.043052 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f513aec-886b-40dc-9e66-0d10a2c8d6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=743ade8649c241b4b43a0af5c9b96365, user_id=7388bc9cb6b64571bd25c3b6f6c2172b, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:43.046008 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f513aec-886b-40dc-9e66-0d10a2c8d6d5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:43.108212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1241/2477] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:07:43 2022] PUT /identity/v3/projects/743ade8649c241b4b43a0af5c9b96365/users/7388bc9cb6b64571bd25c3b6f6c2172b/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:43.112735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cab61-dbd1-4b61-a2c5-6eb0e9f36d08 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:43.123216 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cab61-dbd1-4b61-a2c5-6eb0e9f36d08 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:43.193021 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06a893-a9c7-4a1b-8a0e-9b7e71ee2359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:43.193715 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06a893-a9c7-4a1b-8a0e-9b7e71ee2359 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:43.194024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06a893-a9c7-4a1b-8a0e-9b7e71ee2359 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:43.194812 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06a893-a9c7-4a1b-8a0e-9b7e71ee2359 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:43.195579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06a893-a9c7-4a1b-8a0e-9b7e71ee2359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:43.197176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06a893-a9c7-4a1b-8a0e-9b7e71ee2359 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:43.207003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1237/2478] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:43 2022] GET /identity/v3/roles => generated 1427 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:43.211444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f513aec-886b-40dc-9e66-0d10a2c8d6d5 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:43.222034 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f513aec-886b-40dc-9e66-0d10a2c8d6d5 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:43.227123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd51ea-e756-4715-b31b-35604d356ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:43.228016 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd51ea-e756-4715-b31b-35604d356ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:43.228300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd51ea-e756-4715-b31b-35604d356ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:43.228578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd51ea-e756-4715-b31b-35604d356ff3 admin admin] PATH_INFO: `/v3/projects/743ade8649c241b4b43a0af5c9b96365/users/7388bc9cb6b64571bd25c3b6f6c2172b/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:43.230534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd51ea-e756-4715-b31b-35604d356ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=743ade8649c241b4b43a0af5c9b96365, user_id=7388bc9cb6b64571bd25c3b6f6c2172b, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:43.233214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd51ea-e756-4715-b31b-35604d356ff3 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:43.244623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1242/2479] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:07:43 2022] PUT /identity/v3/projects/743ade8649c241b4b43a0af5c9b96365/users/7388bc9cb6b64571bd25c3b6f6c2172b/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:43.254494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeebe2b-ea65-45d0-8147-7ab48ca06c24 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:43.254806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeebe2b-ea65-45d0-8147-7ab48ca06c24 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:43.255110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeebe2b-ea65-45d0-8147-7ab48ca06c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:43.284052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3aeebe2b-ea65-45d0-8147-7ab48ca06c24 None None] MFA Rules not processed for user `7388bc9cb6b64571bd25c3b6f6c2172b`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:43.285247 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3aeebe2b-ea65-45d0-8147-7ab48ca06c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:43.313095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:43.361681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1238/2480] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:43.374520 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd51ea-e756-4715-b31b-35604d356ff3 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:43.376686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-06cd51ea-e756-4715-b31b-35604d356ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:43.447803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd51ea-e756-4715-b31b-35604d356ff3 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:43.451429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3063a31-5cae-4d7d-b020-c5ca7a8dff15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:43.452108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3063a31-5cae-4d7d-b020-c5ca7a8dff15 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:43.452387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3063a31-5cae-4d7d-b020-c5ca7a8dff15 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:43.452658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3063a31-5cae-4d7d-b020-c5ca7a8dff15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:43.466307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:43.515625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3063a31-5cae-4d7d-b020-c5ca7a8dff15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:43.520113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3063a31-5cae-4d7d-b020-c5ca7a8dff15 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:43.531256 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1243/2481] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:43.719081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeebe2b-ea65-45d0-8147-7ab48ca06c24 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:43.720463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3aeebe2b-ea65-45d0-8147-7ab48ca06c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:43.801397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeebe2b-ea65-45d0-8147-7ab48ca06c24 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:43.807203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb9080-50b9-42fc-84a9-4d9b312e14e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:43.809629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb9080-50b9-42fc-84a9-4d9b312e14e2 service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:43.809629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb9080-50b9-42fc-84a9-4d9b312e14e2 service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:43.809629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb9080-50b9-42fc-84a9-4d9b312e14e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:43.810151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cadb9080-50b9-42fc-84a9-4d9b312e14e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:43.822342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb9080-50b9-42fc-84a9-4d9b312e14e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:43.823916 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb9080-50b9-42fc-84a9-4d9b312e14e2 service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:43.834220 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1239/2482] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:45.647490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3063a31-5cae-4d7d-b020-c5ca7a8dff15 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:45.649140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d3063a31-5cae-4d7d-b020-c5ca7a8dff15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:45.660366 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3063a31-5cae-4d7d-b020-c5ca7a8dff15 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:45.664804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cceb0b-f34b-4dbd-9a0b-2289f0f5ab00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:45.665455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cceb0b-f34b-4dbd-9a0b-2289f0f5ab00 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:45.665726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cceb0b-f34b-4dbd-9a0b-2289f0f5ab00 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:45.666011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cceb0b-f34b-4dbd-9a0b-2289f0f5ab00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:45.721613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cceb0b-f34b-4dbd-9a0b-2289f0f5ab00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:45.723071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cceb0b-f34b-4dbd-9a0b-2289f0f5ab00 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:45.733001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1244/2483] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:47.332747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb9080-50b9-42fc-84a9-4d9b312e14e2 service placement] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:47.342789 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb9080-50b9-42fc-84a9-4d9b312e14e2 service placement] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:47.346361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61c85a-fe72-40a3-93e6-64b89fdec35a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:47.347035 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61c85a-fe72-40a3-93e6-64b89fdec35a service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:47.347319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61c85a-fe72-40a3-93e6-64b89fdec35a service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:47.347595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61c85a-fe72-40a3-93e6-64b89fdec35a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:47.356484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61c85a-fe72-40a3-93e6-64b89fdec35a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:47.357938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61c85a-fe72-40a3-93e6-64b89fdec35a service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:47.367427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1240/2484] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:52.027191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cceb0b-f34b-4dbd-9a0b-2289f0f5ab00 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:52.090208 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cceb0b-f34b-4dbd-9a0b-2289f0f5ab00 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:52.094769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:52.095728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:52.096012 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:52.096287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] PATH_INFO: `/v3/users/7d6a3659c7b34c9db600893370b99205` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:52.097470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6a3659c7b34c9db600893370b99205)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:52.099444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:52.332323 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6a3659c7b34c9db600893370b99205'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:52.340540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6a3659c7b34c9db600893370b99205'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:52.341270 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6a3659c7b34c9db600893370b99205'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:52.342466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:52.353875 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6a3659c7b34c9db600893370b99205'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:52.437778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6a3659c7b34c9db600893370b99205'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:52.444074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1245/2485] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:07:52 2022] DELETE /identity/v3/users/7d6a3659c7b34c9db600893370b99205 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:52.449196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91075ee4-d92b-4d92-968b-e3dda85cee3e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:52.449704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91075ee4-d92b-4d92-968b-e3dda85cee3e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:52.449800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91075ee4-d92b-4d92-968b-e3dda85cee3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:52.466984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-91075ee4-d92b-4d92-968b-e3dda85cee3e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:52.470657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-91075ee4-d92b-4d92-968b-e3dda85cee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:52.531223 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1241/2486] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:52.541880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:52.545130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:52.602262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83093f9f-87b7-4f91-bc39-f782e5095a1e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:52.605983 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0d783-0ccb-4c52-851e-e4e00619b383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:52.606624 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0d783-0ccb-4c52-851e-e4e00619b383 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:52.606954 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0d783-0ccb-4c52-851e-e4e00619b383 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:52.607229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0d783-0ccb-4c52-851e-e4e00619b383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:52.616503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0d783-0ccb-4c52-851e-e4e00619b383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:52.617926 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0d783-0ccb-4c52-851e-e4e00619b383 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:52.627430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1246/2487] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:53.063268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91075ee4-d92b-4d92-968b-e3dda85cee3e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:53.066177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-91075ee4-d92b-4d92-968b-e3dda85cee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:53.082920 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91075ee4-d92b-4d92-968b-e3dda85cee3e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:53.088283 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:53.089152 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:53.089435 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:53.089774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] PATH_INFO: `/v3/projects/b01e92dde7fc49b79f4b7c9d3444a4a8` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:53.090893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01e92dde7fc49b79f4b7c9d3444a4a8)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:53.093008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:53.177940 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:53.188438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01e92dde7fc49b79f4b7c9d3444a4a8', 'request_id': 'req-c8ba9932-d445-414b-948f-c1f09d2be8cf'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:53.237882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01e92dde7fc49b79f4b7c9d3444a4a8', 'request_id': 'req-c8ba9932-d445-414b-948f-c1f09d2be8cf'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:53.247747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01e92dde7fc49b79f4b7c9d3444a4a8', 'request_id': 'req-c8ba9932-d445-414b-948f-c1f09d2be8cf'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:53.259544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] The token cache is being invalidate because project b01e92dde7fc49b79f4b7c9d3444a4a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:53.259823 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:53.260469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:53.262040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1242/2488] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:07:53 2022] DELETE /identity/v3/projects/b01e92dde7fc49b79f4b7c9d3444a4a8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:53.266231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0d783-0ccb-4c52-851e-e4e00619b383 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:53.275699 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0d783-0ccb-4c52-851e-e4e00619b383 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:53.331539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:53.332410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:53.332686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:53.332951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] PATH_INFO: `/v3/users/f60d742e11934dddb285a4750f746191` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:53.334115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f60d742e11934dddb285a4750f746191)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:53.336074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:53.584334 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f60d742e11934dddb285a4750f746191'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:53.597986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f60d742e11934dddb285a4750f746191'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:53.598995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f60d742e11934dddb285a4750f746191'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:53.618942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f60d742e11934dddb285a4750f746191'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:53.624384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:53.712657 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f60d742e11934dddb285a4750f746191'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:53.719420 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1247/2489] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:07:53 2022] DELETE /identity/v3/users/f60d742e11934dddb285a4750f746191 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:54.125808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:54.130561 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:54.141344 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba9932-d445-414b-948f-c1f09d2be8cf admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:54.200232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:54.201111 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:54.201387 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:54.201677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] PATH_INFO: `/v3/projects/3aca008ae52b4c32bd07ed19da16d37e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:54.202861 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aca008ae52b4c32bd07ed19da16d37e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:54.204770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:54.301237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aca008ae52b4c32bd07ed19da16d37e', 'request_id': 'req-72099a1a-431c-454c-8aa5-dcd8dd72beae'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:54.368779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aca008ae52b4c32bd07ed19da16d37e', 'request_id': 'req-72099a1a-431c-454c-8aa5-dcd8dd72beae'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:54.376056 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aca008ae52b4c32bd07ed19da16d37e', 'request_id': 'req-72099a1a-431c-454c-8aa5-dcd8dd72beae'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:54.388898 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] The token cache is being invalidate because project 3aca008ae52b4c32bd07ed19da16d37e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:54.389193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:54.389867 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:54.391533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1243/2490] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:07:54 2022] DELETE /identity/v3/projects/3aca008ae52b4c32bd07ed19da16d37e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:54.395462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:54.396945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:54.410493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c595d-521d-4ac9-a690-9b833f264779 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:54.469880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea7a49-34c5-42bd-a801-d40a68f5cbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:54.470806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea7a49-34c5-42bd-a801-d40a68f5cbbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:54.471109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea7a49-34c5-42bd-a801-d40a68f5cbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:54.471386 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea7a49-34c5-42bd-a801-d40a68f5cbbf admin admin] PATH_INFO: `/v3/users/7388bc9cb6b64571bd25c3b6f6c2172b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:54.472567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea7a49-34c5-42bd-a801-d40a68f5cbbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7388bc9cb6b64571bd25c3b6f6c2172b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:54.474524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea7a49-34c5-42bd-a801-d40a68f5cbbf admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:54.629374 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-9aea7a49-34c5-42bd-a801-d40a68f5cbbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7388bc9cb6b64571bd25c3b6f6c2172b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:54.637500 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-9aea7a49-34c5-42bd-a801-d40a68f5cbbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7388bc9cb6b64571bd25c3b6f6c2172b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:54.638185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-9aea7a49-34c5-42bd-a801-d40a68f5cbbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7388bc9cb6b64571bd25c3b6f6c2172b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:54.650416 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-9aea7a49-34c5-42bd-a801-d40a68f5cbbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7388bc9cb6b64571bd25c3b6f6c2172b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:54.711239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-9aea7a49-34c5-42bd-a801-d40a68f5cbbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7388bc9cb6b64571bd25c3b6f6c2172b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:54.717696 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1248/2491] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:07:54 2022] DELETE /identity/v3/users/7388bc9cb6b64571bd25c3b6f6c2172b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:55.375719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:55.377239 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:55.390765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72099a1a-431c-454c-8aa5-dcd8dd72beae admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:55.440345 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:55.449619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:55.451900 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:55.451900 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:55.451900 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a admin admin] PATH_INFO: `/v3/projects/743ade8649c241b4b43a0af5c9b96365` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:55.452201 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a admin admin] RBAC: Authorizing `identity:delete_project(project_id=743ade8649c241b4b43a0af5c9b96365)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:55.454089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:55.590600 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '743ade8649c241b4b43a0af5c9b96365', 'request_id': 'req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:55.666497 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '743ade8649c241b4b43a0af5c9b96365', 'request_id': 'req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:55.673278 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '743ade8649c241b4b43a0af5c9b96365', 'request_id': 'req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:55.684839 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a admin admin] The token cache is being invalidate because project 743ade8649c241b4b43a0af5c9b96365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:55.687859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:55.687859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cea9f5dd-ee0c-4b8b-886a-23a56d8ce53a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:55.687859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1244/2492] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:07:55 2022] DELETE /identity/v3/projects/743ade8649c241b4b43a0af5c9b96365 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:55.693640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c6ded-3dff-4fff-a360-423b0c3a4c8c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:55.694126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c6ded-3dff-4fff-a360-423b0c3a4c8c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:55.694224 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c6ded-3dff-4fff-a360-423b0c3a4c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:55.712401 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-789c6ded-3dff-4fff-a360-423b0c3a4c8c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:55.713590 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-789c6ded-3dff-4fff-a360-423b0c3a4c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:55.770827 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1249/2493] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:55.779616 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c75b5b-0919-4892-b669-126afe820bc3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:55.779899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c75b5b-0919-4892-b669-126afe820bc3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:55.780171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c75b5b-0919-4892-b669-126afe820bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:55.798896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-85c75b5b-0919-4892-b669-126afe820bc3 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:55.800084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-85c75b5b-0919-4892-b669-126afe820bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:55.808349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1245/2494] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:55.812532 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c6ded-3dff-4fff-a360-423b0c3a4c8c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:55.822070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c6ded-3dff-4fff-a360-423b0c3a4c8c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:55.826701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a78d9-adab-4cfa-9202-51ccbd20b089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:55.827387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a78d9-adab-4cfa-9202-51ccbd20b089 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:55.827666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a78d9-adab-4cfa-9202-51ccbd20b089 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:55.827940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a78d9-adab-4cfa-9202-51ccbd20b089 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:55.828648 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a78d9-adab-4cfa-9202-51ccbd20b089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:55.828951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a78d9-adab-4cfa-9202-51ccbd20b089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:55.830521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a78d9-adab-4cfa-9202-51ccbd20b089 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:55.842252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1250/2495] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:07:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:55.846331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c75b5b-0919-4892-b669-126afe820bc3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:55.857070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c75b5b-0919-4892-b669-126afe820bc3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:55.861648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eaf35d-031a-4a6e-bc67-ffa90a7aa0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:55.862361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eaf35d-031a-4a6e-bc67-ffa90a7aa0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:55.862667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eaf35d-031a-4a6e-bc67-ffa90a7aa0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:55.862985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eaf35d-031a-4a6e-bc67-ffa90a7aa0b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:55.863730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eaf35d-031a-4a6e-bc67-ffa90a7aa0b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:55.865635 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eaf35d-031a-4a6e-bc67-ffa90a7aa0b5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:55.988187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1246/2496] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:07:55 2022] POST /identity/v3/projects => generated 364 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:55.992230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a78d9-adab-4cfa-9202-51ccbd20b089 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:55.997481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:56.002752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a78d9-adab-4cfa-9202-51ccbd20b089 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:56.061422 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb1366-05c4-409b-99eb-8d1bebb102b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:56.062090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb1366-05c4-409b-99eb-8d1bebb102b8 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:56.062371 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb1366-05c4-409b-99eb-8d1bebb102b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:56.062643 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb1366-05c4-409b-99eb-8d1bebb102b8 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:56.063363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb1366-05c4-409b-99eb-8d1bebb102b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:56.065260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb1366-05c4-409b-99eb-8d1bebb102b8 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:56.187111 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1251/2497] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:07:55 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:56.191090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eaf35d-031a-4a6e-bc67-ffa90a7aa0b5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:56.201926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eaf35d-031a-4a6e-bc67-ffa90a7aa0b5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:56.206377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431849d-b4ea-4594-badb-6f7de9ebcfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:56.207101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431849d-b4ea-4594-badb-6f7de9ebcfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:56.207376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431849d-b4ea-4594-badb-6f7de9ebcfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:56.207642 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431849d-b4ea-4594-badb-6f7de9ebcfc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:56.208356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431849d-b4ea-4594-badb-6f7de9ebcfc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:56.209932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431849d-b4ea-4594-badb-6f7de9ebcfc1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:56.220268 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1247/2498] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:56 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:56.224305 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb1366-05c4-409b-99eb-8d1bebb102b8 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:56.235431 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb1366-05c4-409b-99eb-8d1bebb102b8 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:56.240092 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f729033-574f-4b86-808a-2f3e33d277f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:56.240887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f729033-574f-4b86-808a-2f3e33d277f9 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:56.241177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f729033-574f-4b86-808a-2f3e33d277f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:56.241458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f729033-574f-4b86-808a-2f3e33d277f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:56.242184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f729033-574f-4b86-808a-2f3e33d277f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:56.244725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f729033-574f-4b86-808a-2f3e33d277f9 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:56.255031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1252/2499] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:07:56 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:56.259765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431849d-b4ea-4594-badb-6f7de9ebcfc1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:56.269579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431849d-b4ea-4594-badb-6f7de9ebcfc1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:56.274027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565a9ba-8d80-43b9-96f8-62b2ea400ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:56.274932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565a9ba-8d80-43b9-96f8-62b2ea400ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:56.275211 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565a9ba-8d80-43b9-96f8-62b2ea400ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:56.275482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565a9ba-8d80-43b9-96f8-62b2ea400ee9 admin admin] PATH_INFO: `/v3/projects/c62c1ea7b0e74bf8b0f311fbb1e6d93d/users/bb7abd1e1da946aba9d22c2fd62d0c31/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:56.290574 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565a9ba-8d80-43b9-96f8-62b2ea400ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62c1ea7b0e74bf8b0f311fbb1e6d93d, user_id=bb7abd1e1da946aba9d22c2fd62d0c31, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:56.293354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565a9ba-8d80-43b9-96f8-62b2ea400ee9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:56.374626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1248/2500] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:07:56 2022] PUT /identity/v3/projects/c62c1ea7b0e74bf8b0f311fbb1e6d93d/users/bb7abd1e1da946aba9d22c2fd62d0c31/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:56.385793 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de16a9-abab-4de1-ba25-f9be718fa305 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:56.386083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de16a9-abab-4de1-ba25-f9be718fa305 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:56.386493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de16a9-abab-4de1-ba25-f9be718fa305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:56.412924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-93de16a9-abab-4de1-ba25-f9be718fa305 None None] MFA Rules not processed for user `bb7abd1e1da946aba9d22c2fd62d0c31`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:56.414107 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-93de16a9-abab-4de1-ba25-f9be718fa305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:56.478404 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1253/2501] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:07:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:56.489601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565a9ba-8d80-43b9-96f8-62b2ea400ee9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:56.491370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a565a9ba-8d80-43b9-96f8-62b2ea400ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:56.547785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565a9ba-8d80-43b9-96f8-62b2ea400ee9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:56.551855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa054b-7cdf-406a-9223-e670e87cbb20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:56.552496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa054b-7cdf-406a-9223-e670e87cbb20 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:56.552773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa054b-7cdf-406a-9223-e670e87cbb20 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:56.553045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa054b-7cdf-406a-9223-e670e87cbb20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:56.561523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa054b-7cdf-406a-9223-e670e87cbb20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:56.563048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa054b-7cdf-406a-9223-e670e87cbb20 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:56.572340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1249/2502] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:07:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:08:06.475193 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:06.567824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:33.278830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae92e8-0191-47c9-b67b-ce0969ebcb8a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:33.279383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae92e8-0191-47c9-b67b-ce0969ebcb8a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:33.279383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae92e8-0191-47c9-b67b-ce0969ebcb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:33.297342 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-11ae92e8-0191-47c9-b67b-ce0969ebcb8a None None] MFA Rules not processed for user `4ffcb99923b346938eb72c0b2786e815`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:08:33.298531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-11ae92e8-0191-47c9-b67b-ce0969ebcb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:33.359219 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1254/2503] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:08:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:08:43.351898 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=25 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:46.444292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa054b-7cdf-406a-9223-e670e87cbb20 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:46.507551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa054b-7cdf-406a-9223-e670e87cbb20 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:46.512544 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:46.513414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:46.513695 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:46.513968 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] PATH_INFO: `/v3/users/bb7abd1e1da946aba9d22c2fd62d0c31` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:46.515229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb7abd1e1da946aba9d22c2fd62d0c31)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:46.517155 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:46.737047 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb7abd1e1da946aba9d22c2fd62d0c31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:08:46.744124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb7abd1e1da946aba9d22c2fd62d0c31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:08:46.744787 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb7abd1e1da946aba9d22c2fd62d0c31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:08:46.756822 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb7abd1e1da946aba9d22c2fd62d0c31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:08:46.860304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb7abd1e1da946aba9d22c2fd62d0c31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:08:46.866699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1250/2504] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:08:46 2022] DELETE /identity/v3/users/bb7abd1e1da946aba9d22c2fd62d0c31 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:08:46.871179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bf034-2615-45e0-8dc3-dbc257038e14 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:46.871686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bf034-2615-45e0-8dc3-dbc257038e14 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:46.871787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bf034-2615-45e0-8dc3-dbc257038e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:46.891165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5c6bf034-2615-45e0-8dc3-dbc257038e14 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:08:46.892357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5c6bf034-2615-45e0-8dc3-dbc257038e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:46.949627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1255/2505] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:08:46.959908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:46.961298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:47.018368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eab98d-9276-4cb9-9a12-89b25ef41bf3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:47.021860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbc21e-63fa-481b-8692-f14548b4130a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:47.022502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc21e-63fa-481b-8692-f14548b4130a service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:47.022782 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc21e-63fa-481b-8692-f14548b4130a service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:47.023091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc21e-63fa-481b-8692-f14548b4130a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:47.032125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbc21e-63fa-481b-8692-f14548b4130a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:47.033547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbc21e-63fa-481b-8692-f14548b4130a service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:47.042799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1251/2506] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:08:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:08:47.571942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bf034-2615-45e0-8dc3-dbc257038e14 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:47.574085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5c6bf034-2615-45e0-8dc3-dbc257038e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:47.589739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bf034-2615-45e0-8dc3-dbc257038e14 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:47.603021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec30d7-241d-49d6-9680-b9fadc894823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:47.603021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec30d7-241d-49d6-9680-b9fadc894823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:47.603021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec30d7-241d-49d6-9680-b9fadc894823 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:47.603021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec30d7-241d-49d6-9680-b9fadc894823 admin admin] PATH_INFO: `/v3/projects/c62c1ea7b0e74bf8b0f311fbb1e6d93d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:47.604369 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec30d7-241d-49d6-9680-b9fadc894823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c62c1ea7b0e74bf8b0f311fbb1e6d93d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:47.605055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec30d7-241d-49d6-9680-b9fadc894823 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:47.748125 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-faec30d7-241d-49d6-9680-b9fadc894823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c62c1ea7b0e74bf8b0f311fbb1e6d93d', 'request_id': 'req-faec30d7-241d-49d6-9680-b9fadc894823'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:08:47.837757 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-faec30d7-241d-49d6-9680-b9fadc894823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c62c1ea7b0e74bf8b0f311fbb1e6d93d', 'request_id': 'req-faec30d7-241d-49d6-9680-b9fadc894823'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:08:47.844962 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-faec30d7-241d-49d6-9680-b9fadc894823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c62c1ea7b0e74bf8b0f311fbb1e6d93d', 'request_id': 'req-faec30d7-241d-49d6-9680-b9fadc894823'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:08:47.855528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-faec30d7-241d-49d6-9680-b9fadc894823 admin admin] The token cache is being invalidate because project c62c1ea7b0e74bf8b0f311fbb1e6d93d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:08:47.855814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-faec30d7-241d-49d6-9680-b9fadc894823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:08:47.856455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-faec30d7-241d-49d6-9680-b9fadc894823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:08:47.858648 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1256/2507] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:08:47 2022] DELETE /identity/v3/projects/c62c1ea7b0e74bf8b0f311fbb1e6d93d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:08:47.863602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2d5be-6374-4eba-9b20-ea1d9292c621 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:47.864073 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2d5be-6374-4eba-9b20-ea1d9292c621 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:47.864196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2d5be-6374-4eba-9b20-ea1d9292c621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:47.881947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2ae2d5be-6374-4eba-9b20-ea1d9292c621 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:08:47.883185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2ae2d5be-6374-4eba-9b20-ea1d9292c621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:47.901724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:47.944330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1252/2508] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:08:47.953241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de37e0-3a29-4947-b918-2a7dc08bab8c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:47.953786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de37e0-3a29-4947-b918-2a7dc08bab8c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:47.953786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de37e0-3a29-4947-b918-2a7dc08bab8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:47.973264 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-82de37e0-3a29-4947-b918-2a7dc08bab8c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:08:47.974608 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-82de37e0-3a29-4947-b918-2a7dc08bab8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:47.983177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1257/2509] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:08:47.987402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2d5be-6374-4eba-9b20-ea1d9292c621 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:47.997219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2d5be-6374-4eba-9b20-ea1d9292c621 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:48.001866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80ff4e-a44f-41d3-ada4-69bfb776d74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:48.002581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80ff4e-a44f-41d3-ada4-69bfb776d74c admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:48.002897 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80ff4e-a44f-41d3-ada4-69bfb776d74c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:48.003171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80ff4e-a44f-41d3-ada4-69bfb776d74c admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:48.003901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80ff4e-a44f-41d3-ada4-69bfb776d74c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:08:48.004206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80ff4e-a44f-41d3-ada4-69bfb776d74c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:48.005808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80ff4e-a44f-41d3-ada4-69bfb776d74c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:48.018139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1253/2510] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:08:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:08:48.022363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de37e0-3a29-4947-b918-2a7dc08bab8c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:48.032385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de37e0-3a29-4947-b918-2a7dc08bab8c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:48.037113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffc6f9-fd44-474b-9db4-652b5bdc8a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:48.037766 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffc6f9-fd44-474b-9db4-652b5bdc8a4c admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:48.038064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffc6f9-fd44-474b-9db4-652b5bdc8a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:48.038336 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffc6f9-fd44-474b-9db4-652b5bdc8a4c admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:48.039057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffc6f9-fd44-474b-9db4-652b5bdc8a4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:48.040938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffc6f9-fd44-474b-9db4-652b5bdc8a4c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:48.160797 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1258/2511] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:08:48 2022] POST /identity/v3/projects => generated 390 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:08:48.164913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80ff4e-a44f-41d3-ada4-69bfb776d74c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:48.175553 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80ff4e-a44f-41d3-ada4-69bfb776d74c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:48.233342 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b633665-530e-4582-9afb-90090978257a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:48.234016 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b633665-530e-4582-9afb-90090978257a admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:48.234296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b633665-530e-4582-9afb-90090978257a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:48.234570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b633665-530e-4582-9afb-90090978257a admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:48.235291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b633665-530e-4582-9afb-90090978257a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:48.237197 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b633665-530e-4582-9afb-90090978257a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:48.364367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1254/2512] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:08:48 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:08:48.368538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffc6f9-fd44-474b-9db4-652b5bdc8a4c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:48.379847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffc6f9-fd44-474b-9db4-652b5bdc8a4c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:48.384377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e5b73-ff8e-401c-b09e-8904ae63b0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:48.385038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e5b73-ff8e-401c-b09e-8904ae63b0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:48.385320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e5b73-ff8e-401c-b09e-8904ae63b0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:48.385594 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e5b73-ff8e-401c-b09e-8904ae63b0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:48.386310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e5b73-ff8e-401c-b09e-8904ae63b0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:48.387934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e5b73-ff8e-401c-b09e-8904ae63b0c6 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:48.399475 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1259/2513] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:08:48 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:08:48.403423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b633665-530e-4582-9afb-90090978257a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:48.413551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b633665-530e-4582-9afb-90090978257a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:48.418146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5769ce-0f3c-477f-9e64-9043b7b1039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:48.418801 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5769ce-0f3c-477f-9e64-9043b7b1039a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:48.419115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5769ce-0f3c-477f-9e64-9043b7b1039a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:48.419388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5769ce-0f3c-477f-9e64-9043b7b1039a admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:48.420105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5769ce-0f3c-477f-9e64-9043b7b1039a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:48.421689 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5769ce-0f3c-477f-9e64-9043b7b1039a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:48.431976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1255/2514] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:08:48 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:08:48.436744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e5b73-ff8e-401c-b09e-8904ae63b0c6 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:48.446932 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e5b73-ff8e-401c-b09e-8904ae63b0c6 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:48.452907 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc0a3b-1525-4003-a83a-991841e5da88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:48.453809 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc0a3b-1525-4003-a83a-991841e5da88 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:48.454090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc0a3b-1525-4003-a83a-991841e5da88 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:48.454774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc0a3b-1525-4003-a83a-991841e5da88 admin admin] PATH_INFO: `/v3/projects/5e5841a4b95741308a00538d6ccf4f6f/users/1dafadc42b24422186d00e896cd963f2/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:48.472339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc0a3b-1525-4003-a83a-991841e5da88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e5841a4b95741308a00538d6ccf4f6f, user_id=1dafadc42b24422186d00e896cd963f2, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:48.475205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc0a3b-1525-4003-a83a-991841e5da88 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:48.577105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1260/2515] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:08:48 2022] PUT /identity/v3/projects/5e5841a4b95741308a00538d6ccf4f6f/users/1dafadc42b24422186d00e896cd963f2/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:08:48.588319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91ea96-f937-42ea-b82b-480e9b1f675b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:48.588729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91ea96-f937-42ea-b82b-480e9b1f675b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:48.589082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91ea96-f937-42ea-b82b-480e9b1f675b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:48.617138 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7b91ea96-f937-42ea-b82b-480e9b1f675b None None] MFA Rules not processed for user `1dafadc42b24422186d00e896cd963f2`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:08:48.618354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7b91ea96-f937-42ea-b82b-480e9b1f675b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:48.693853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1256/2516] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:08:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:08:48.705025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc0a3b-1525-4003-a83a-991841e5da88 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:48.706677 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b6dc0a3b-1525-4003-a83a-991841e5da88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:48.729168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:48.766694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc0a3b-1525-4003-a83a-991841e5da88 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:48.771750 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcb158-d07d-4338-badf-f8491ac15021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:48.772406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcb158-d07d-4338-badf-f8491ac15021 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:48.772690 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcb158-d07d-4338-badf-f8491ac15021 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:48.772958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcb158-d07d-4338-badf-f8491ac15021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:48.781760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcb158-d07d-4338-badf-f8491ac15021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:48.783225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcb158-d07d-4338-badf-f8491ac15021 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:48.792663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1261/2517] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:08:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:08:58.690507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:58.820734 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=17 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:01.640867 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fbaf6-600b-4733-ab4a-7d64de486b23 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:01.642257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fbaf6-600b-4733-ab4a-7d64de486b23 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:01.642257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fbaf6-600b-4733-ab4a-7d64de486b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:01.660891 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-611fbaf6-600b-4733-ab4a-7d64de486b23 None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:01.662072 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-611fbaf6-600b-4733-ab4a-7d64de486b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:01.725096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1257/2518] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 18:09:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:01.858235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcb158-d07d-4338-badf-f8491ac15021 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:01.859693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-91fcb158-d07d-4338-badf-f8491ac15021 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:01.929159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcb158-d07d-4338-badf-f8491ac15021 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:01.935579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39cea2-1da3-477e-98bd-f75d840bfca1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:01.936230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39cea2-1da3-477e-98bd-f75d840bfca1 service swift] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:01.936508 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39cea2-1da3-477e-98bd-f75d840bfca1 service swift] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:01.936779 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39cea2-1da3-477e-98bd-f75d840bfca1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:01.945057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39cea2-1da3-477e-98bd-f75d840bfca1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:01.947126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39cea2-1da3-477e-98bd-f75d840bfca1 service swift] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:01.955441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1262/2519] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:09:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:09:11.718441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=25 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:11.952179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=32 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:15.386716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fbaf6-600b-4733-ab4a-7d64de486b23 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:15.448184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fbaf6-600b-4733-ab4a-7d64de486b23 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:15.452610 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:15.454039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:15.454315 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:15.454582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] PATH_INFO: `/v3/users/1dafadc42b24422186d00e896cd963f2` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:15.455791 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dafadc42b24422186d00e896cd963f2)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:15.457693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:15.616735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dafadc42b24422186d00e896cd963f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:09:15.624005 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dafadc42b24422186d00e896cd963f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:09:15.625312 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dafadc42b24422186d00e896cd963f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:09:15.637082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dafadc42b24422186d00e896cd963f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:09:15.711340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dafadc42b24422186d00e896cd963f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:09:15.718067 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1258/2520] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:09:15 2022] DELETE /identity/v3/users/1dafadc42b24422186d00e896cd963f2 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:09:15.737324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f113331-9857-44a2-ada7-4ef7fcd3f558 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:15.737324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f113331-9857-44a2-ada7-4ef7fcd3f558 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:15.737324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f113331-9857-44a2-ada7-4ef7fcd3f558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:15.741118 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3f113331-9857-44a2-ada7-4ef7fcd3f558 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:15.742662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3f113331-9857-44a2-ada7-4ef7fcd3f558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:15.803942 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1263/2521] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:15.813586 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:15.815029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:15.874465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54890944-022b-4527-b261-9e50abea7514 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:15.878507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f81768-7151-4090-ae67-fa67aa471c11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:15.879260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f81768-7151-4090-ae67-fa67aa471c11 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:15.879543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f81768-7151-4090-ae67-fa67aa471c11 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:15.879817 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f81768-7151-4090-ae67-fa67aa471c11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:15.889507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f81768-7151-4090-ae67-fa67aa471c11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:15.891650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f81768-7151-4090-ae67-fa67aa471c11 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:15.901958 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1259/2522] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:09:16.349074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f113331-9857-44a2-ada7-4ef7fcd3f558 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:16.353577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3f113331-9857-44a2-ada7-4ef7fcd3f558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:16.371387 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f113331-9857-44a2-ada7-4ef7fcd3f558 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:16.379844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cefff8-d498-48ee-b351-2ea6582640c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:16.381378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cefff8-d498-48ee-b351-2ea6582640c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:16.381885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cefff8-d498-48ee-b351-2ea6582640c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:16.382388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cefff8-d498-48ee-b351-2ea6582640c2 admin admin] PATH_INFO: `/v3/projects/5e5841a4b95741308a00538d6ccf4f6f` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:16.384044 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cefff8-d498-48ee-b351-2ea6582640c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e5841a4b95741308a00538d6ccf4f6f)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:16.387191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cefff8-d498-48ee-b351-2ea6582640c2 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:16.535103 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50cefff8-d498-48ee-b351-2ea6582640c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e5841a4b95741308a00538d6ccf4f6f', 'request_id': 'req-50cefff8-d498-48ee-b351-2ea6582640c2'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:09:16.654929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50cefff8-d498-48ee-b351-2ea6582640c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e5841a4b95741308a00538d6ccf4f6f', 'request_id': 'req-50cefff8-d498-48ee-b351-2ea6582640c2'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:09:16.662881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50cefff8-d498-48ee-b351-2ea6582640c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e5841a4b95741308a00538d6ccf4f6f', 'request_id': 'req-50cefff8-d498-48ee-b351-2ea6582640c2'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:09:16.673211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50cefff8-d498-48ee-b351-2ea6582640c2 admin admin] The token cache is being invalidate because project 5e5841a4b95741308a00538d6ccf4f6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:09:16.673552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50cefff8-d498-48ee-b351-2ea6582640c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:09:16.674281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50cefff8-d498-48ee-b351-2ea6582640c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:09:16.676144 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1264/2523] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:09:16 2022] DELETE /identity/v3/projects/5e5841a4b95741308a00538d6ccf4f6f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:09:16.681181 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e71883-1fd8-44e0-8c54-9293c7dbbb22 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:16.681663 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e71883-1fd8-44e0-8c54-9293c7dbbb22 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:16.681774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e71883-1fd8-44e0-8c54-9293c7dbbb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:16.699657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e5e71883-1fd8-44e0-8c54-9293c7dbbb22 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:16.700886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e5e71883-1fd8-44e0-8c54-9293c7dbbb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:16.718204 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:16.760779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1260/2524] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:16.769358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e713e89-1d12-44f5-9406-d94a4631b9d6 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:16.769864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e713e89-1d12-44f5-9406-d94a4631b9d6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:16.770061 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e713e89-1d12-44f5-9406-d94a4631b9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:16.788679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5e713e89-1d12-44f5-9406-d94a4631b9d6 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:16.789874 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5e713e89-1d12-44f5-9406-d94a4631b9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:16.797733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1265/2525] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:09:16.802888 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e71883-1fd8-44e0-8c54-9293c7dbbb22 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:16.812383 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e71883-1fd8-44e0-8c54-9293c7dbbb22 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:16.818008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98206ed4-64f2-4f67-81f0-87773b4aa62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:16.818667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98206ed4-64f2-4f67-81f0-87773b4aa62b admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:16.818994 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98206ed4-64f2-4f67-81f0-87773b4aa62b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:16.821463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98206ed4-64f2-4f67-81f0-87773b4aa62b admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:16.821463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98206ed4-64f2-4f67-81f0-87773b4aa62b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:09:16.821463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98206ed4-64f2-4f67-81f0-87773b4aa62b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:16.821847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98206ed4-64f2-4f67-81f0-87773b4aa62b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:16.833632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1261/2526] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:09:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:09:16.838141 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e713e89-1d12-44f5-9406-d94a4631b9d6 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:16.847842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e713e89-1d12-44f5-9406-d94a4631b9d6 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:16.852542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e163139-1622-4e0b-88bd-3e8194512371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:16.853189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e163139-1622-4e0b-88bd-3e8194512371 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:16.853470 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e163139-1622-4e0b-88bd-3e8194512371 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:16.853814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e163139-1622-4e0b-88bd-3e8194512371 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:16.854495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e163139-1622-4e0b-88bd-3e8194512371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:16.856429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e163139-1622-4e0b-88bd-3e8194512371 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:16.953713 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1266/2527] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:09:16 2022] POST /identity/v3/projects => generated 380 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:09:16.958060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98206ed4-64f2-4f67-81f0-87773b4aa62b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:16.968168 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98206ed4-64f2-4f67-81f0-87773b4aa62b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:17.038076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeec267-60ab-4061-9bb4-344a1c83eaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:17.038766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeec267-60ab-4061-9bb4-344a1c83eaee admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:17.039072 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeec267-60ab-4061-9bb4-344a1c83eaee admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:17.039341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeec267-60ab-4061-9bb4-344a1c83eaee admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:17.040558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeec267-60ab-4061-9bb4-344a1c83eaee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:17.042876 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeec267-60ab-4061-9bb4-344a1c83eaee admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:17.119101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1262/2528] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:09:16 2022] POST /identity/v3/users => generated 291 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:09:17.124604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e163139-1622-4e0b-88bd-3e8194512371 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:17.134201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e163139-1622-4e0b-88bd-3e8194512371 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:17.138746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590f171-cb3c-4161-a5bb-ba67729f1aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:17.139574 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590f171-cb3c-4161-a5bb-ba67729f1aaf admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:17.139862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590f171-cb3c-4161-a5bb-ba67729f1aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:17.140200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590f171-cb3c-4161-a5bb-ba67729f1aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:17.140850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590f171-cb3c-4161-a5bb-ba67729f1aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:17.142445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590f171-cb3c-4161-a5bb-ba67729f1aaf admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:17.158676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1267/2529] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:09:17 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:09:17.162804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeec267-60ab-4061-9bb4-344a1c83eaee admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:17.173200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeec267-60ab-4061-9bb4-344a1c83eaee admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:17.178864 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c742805-e317-48c4-a30a-f07cbd5b40bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:17.179624 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c742805-e317-48c4-a30a-f07cbd5b40bd admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:17.179999 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c742805-e317-48c4-a30a-f07cbd5b40bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:17.180340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c742805-e317-48c4-a30a-f07cbd5b40bd admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:17.181266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c742805-e317-48c4-a30a-f07cbd5b40bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:17.183045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c742805-e317-48c4-a30a-f07cbd5b40bd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:17.194023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1263/2530] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:09:17 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:09:17.198396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590f171-cb3c-4161-a5bb-ba67729f1aaf admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:17.209208 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590f171-cb3c-4161-a5bb-ba67729f1aaf admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:17.214570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d9c9b-a31d-4338-b8e4-366ba169776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:17.215323 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d9c9b-a31d-4338-b8e4-366ba169776e admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:17.215674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d9c9b-a31d-4338-b8e4-366ba169776e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:17.216028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d9c9b-a31d-4338-b8e4-366ba169776e admin admin] PATH_INFO: `/v3/projects/7daf2a694ab840949d8cf403c2573bda/users/e7e74d9bfa6144ae8ad7e3156fb42e6e/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:17.232140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d9c9b-a31d-4338-b8e4-366ba169776e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7daf2a694ab840949d8cf403c2573bda, user_id=e7e74d9bfa6144ae8ad7e3156fb42e6e, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:17.234863 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d9c9b-a31d-4338-b8e4-366ba169776e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:17.318843 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1268/2531] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:09:17 2022] PUT /identity/v3/projects/7daf2a694ab840949d8cf403c2573bda/users/e7e74d9bfa6144ae8ad7e3156fb42e6e/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:09:17.324062 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfc82f-db95-4ab7-a687-6024a6cf1a9a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:17.324377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfc82f-db95-4ab7-a687-6024a6cf1a9a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:17.324628 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfc82f-db95-4ab7-a687-6024a6cf1a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:17.342881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c6dfc82f-db95-4ab7-a687-6024a6cf1a9a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:17.344080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c6dfc82f-db95-4ab7-a687-6024a6cf1a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:17.410740 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1264/2532] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:17.420272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d9c9b-a31d-4338-b8e4-366ba169776e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:17.422018 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-863d9c9b-a31d-4338-b8e4-366ba169776e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:17.445123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:17.480121 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d9c9b-a31d-4338-b8e4-366ba169776e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:17.483761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da961-bab7-46c6-bf89-dabc070889c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:17.484406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da961-bab7-46c6-bf89-dabc070889c0 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:17.484685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da961-bab7-46c6-bf89-dabc070889c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:17.484956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da961-bab7-46c6-bf89-dabc070889c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:17.494576 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419da961-bab7-46c6-bf89-dabc070889c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:17.496042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419da961-bab7-46c6-bf89-dabc070889c0 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:17.505487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1269/2533] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:09:22.933773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e72ce2-111e-4828-b043-ed06987ef1f2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:22.934327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e72ce2-111e-4828-b043-ed06987ef1f2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:22.934327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e72ce2-111e-4828-b043-ed06987ef1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:22.962765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f4e72ce2-111e-4828-b043-ed06987ef1f2 None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:22.964400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f4e72ce2-111e-4828-b043-ed06987ef1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:23.002137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:23.037940 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1265/2534] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 18:09:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:23.051253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da961-bab7-46c6-bf89-dabc070889c0 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:23.053971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-419da961-bab7-46c6-bf89-dabc070889c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:23.137990 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da961-bab7-46c6-bf89-dabc070889c0 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:23.143943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6657b-cd54-41f7-80d8-bfbfc1c1bc20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:23.144841 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6657b-cd54-41f7-80d8-bfbfc1c1bc20 service swift] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:23.145179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6657b-cd54-41f7-80d8-bfbfc1c1bc20 service swift] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:23.145701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6657b-cd54-41f7-80d8-bfbfc1c1bc20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:23.162157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6657b-cd54-41f7-80d8-bfbfc1c1bc20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:23.162157 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6657b-cd54-41f7-80d8-bfbfc1c1bc20 service swift] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:23.167304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1270/2535] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:09:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:09:24.390003 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97be305-e878-4b89-ba8d-d3b3e3e0a13f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:24.390975 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97be305-e878-4b89-ba8d-d3b3e3e0a13f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:24.391494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97be305-e878-4b89-ba8d-d3b3e3e0a13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:24.434932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e97be305-e878-4b89-ba8d-d3b3e3e0a13f None None] MFA Rules not processed for user `e7e74d9bfa6144ae8ad7e3156fb42e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:24.438210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e97be305-e878-4b89-ba8d-d3b3e3e0a13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:24.541206 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1266/2536] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:09:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:24.542636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6657b-cd54-41f7-80d8-bfbfc1c1bc20 service swift] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:24.561186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6657b-cd54-41f7-80d8-bfbfc1c1bc20 service swift] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:24.567189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe869d-dcf2-435a-b11c-ccf84c66b36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:24.568217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe869d-dcf2-435a-b11c-ccf84c66b36a admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:24.568615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe869d-dcf2-435a-b11c-ccf84c66b36a admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:24.569647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe869d-dcf2-435a-b11c-ccf84c66b36a admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:24.570475 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe869d-dcf2-435a-b11c-ccf84c66b36a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:24.573244 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe869d-dcf2-435a-b11c-ccf84c66b36a admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:24.715561 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1271/2537] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:09:24 2022] POST /identity/v3/projects => generated 380 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:09:24.719776 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97be305-e878-4b89-ba8d-d3b3e3e0a13f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:24.731068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97be305-e878-4b89-ba8d-d3b3e3e0a13f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:24.855295 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d55bfb-5a9e-49b3-8f56-3b2e2bb6390f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:24.855978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d55bfb-5a9e-49b3-8f56-3b2e2bb6390f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:24.856260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d55bfb-5a9e-49b3-8f56-3b2e2bb6390f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:24.856535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d55bfb-5a9e-49b3-8f56-3b2e2bb6390f admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:24.857230 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d55bfb-5a9e-49b3-8f56-3b2e2bb6390f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:24.859219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d55bfb-5a9e-49b3-8f56-3b2e2bb6390f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:25.022219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1267/2538] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:09:24 2022] POST /identity/v3/users => generated 297 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:09:25.025906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe869d-dcf2-435a-b11c-ccf84c66b36a admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:25.040471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe869d-dcf2-435a-b11c-ccf84c66b36a admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:25.045560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ba282-9855-42e0-a8a1-9db7b70ff7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:25.046519 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ba282-9855-42e0-a8a1-9db7b70ff7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:25.047195 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ba282-9855-42e0-a8a1-9db7b70ff7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:25.047632 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ba282-9855-42e0-a8a1-9db7b70ff7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:25.048613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ba282-9855-42e0-a8a1-9db7b70ff7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:25.051285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ba282-9855-42e0-a8a1-9db7b70ff7d5 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:25.062926 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1272/2539] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:09:25 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:09:25.067980 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d55bfb-5a9e-49b3-8f56-3b2e2bb6390f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:25.095292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d55bfb-5a9e-49b3-8f56-3b2e2bb6390f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:25.105845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39b046-1de6-4af9-b5bd-63bdcc77cd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:25.107101 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39b046-1de6-4af9-b5bd-63bdcc77cd67 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:25.107961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39b046-1de6-4af9-b5bd-63bdcc77cd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:25.109928 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39b046-1de6-4af9-b5bd-63bdcc77cd67 admin admin] PATH_INFO: `/v3/projects/cfc02810c21b45ed8608a00f56ff9fe9/users/2b82ed5b3a2c408a8f8623729062baec/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:25.140907 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39b046-1de6-4af9-b5bd-63bdcc77cd67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc02810c21b45ed8608a00f56ff9fe9, user_id=2b82ed5b3a2c408a8f8623729062baec, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:25.149092 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39b046-1de6-4af9-b5bd-63bdcc77cd67 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:25.241779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1268/2540] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:09:25 2022] PUT /identity/v3/projects/cfc02810c21b45ed8608a00f56ff9fe9/users/2b82ed5b3a2c408a8f8623729062baec/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:09:25.245752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ba282-9855-42e0-a8a1-9db7b70ff7d5 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:25.257227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ba282-9855-42e0-a8a1-9db7b70ff7d5 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:25.347126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8437d-dfe2-4191-a610-0434daec4650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:25.347474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8437d-dfe2-4191-a610-0434daec4650 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:25.347690 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8437d-dfe2-4191-a610-0434daec4650 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:25.347968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8437d-dfe2-4191-a610-0434daec4650 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:25.348698 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8437d-dfe2-4191-a610-0434daec4650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:25.350302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8437d-dfe2-4191-a610-0434daec4650 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:25.363322 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1273/2541] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:09:25 2022] GET /identity/v3/roles => generated 1427 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:09:25.367223 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39b046-1de6-4af9-b5bd-63bdcc77cd67 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:25.377579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39b046-1de6-4af9-b5bd-63bdcc77cd67 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:25.383464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15eae1e-f118-4ad5-96d9-eda6ab874b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:25.384369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eae1e-f118-4ad5-96d9-eda6ab874b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:25.384648 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eae1e-f118-4ad5-96d9-eda6ab874b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:25.384965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eae1e-f118-4ad5-96d9-eda6ab874b20 admin admin] PATH_INFO: `/v3/projects/cfc02810c21b45ed8608a00f56ff9fe9/users/2b82ed5b3a2c408a8f8623729062baec/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:25.387507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15eae1e-f118-4ad5-96d9-eda6ab874b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc02810c21b45ed8608a00f56ff9fe9, user_id=2b82ed5b3a2c408a8f8623729062baec, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:25.390802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15eae1e-f118-4ad5-96d9-eda6ab874b20 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:25.407308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1269/2542] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:09:25 2022] PUT /identity/v3/projects/cfc02810c21b45ed8608a00f56ff9fe9/users/2b82ed5b3a2c408a8f8623729062baec/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:09:33.452814 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4487f-4c37-4e14-a1c6-72b0c6274dfa None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:33.453404 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4487f-4c37-4e14-a1c6-72b0c6274dfa None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:33.453404 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4487f-4c37-4e14-a1c6-72b0c6274dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:33.483423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-03f4487f-4c37-4e14-a1c6-72b0c6274dfa None None] MFA Rules not processed for user `2b82ed5b3a2c408a8f8623729062baec`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:33.484577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-03f4487f-4c37-4e14-a1c6-72b0c6274dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:33.489371 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:33.556230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1274/2543] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:09:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:33.570618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15eae1e-f118-4ad5-96d9-eda6ab874b20 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:33.572446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b15eae1e-f118-4ad5-96d9-eda6ab874b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:33.592956 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:33.651640 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15eae1e-f118-4ad5-96d9-eda6ab874b20 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:33.657288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c7c78-38c4-477f-882f-1654722553ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:33.660747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c7c78-38c4-477f-882f-1654722553ca service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:33.660747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c7c78-38c4-477f-882f-1654722553ca service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:33.660747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c7c78-38c4-477f-882f-1654722553ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:33.713618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c7c78-38c4-477f-882f-1654722553ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:33.715257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c7c78-38c4-477f-882f-1654722553ca service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:33.725301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1270/2544] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:09:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:39.446492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4487f-4c37-4e14-a1c6-72b0c6274dfa None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:39.448013 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-03f4487f-4c37-4e14-a1c6-72b0c6274dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:39.508537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4487f-4c37-4e14-a1c6-72b0c6274dfa None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:39.513001 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82a52c-ba75-4535-908f-be2ff60927d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:39.514442 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82a52c-ba75-4535-908f-be2ff60927d2 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:39.514442 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82a52c-ba75-4535-908f-be2ff60927d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:39.514718 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82a52c-ba75-4535-908f-be2ff60927d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:39.516675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-cc82a52c-ba75-4535-908f-be2ff60927d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:39.532771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82a52c-ba75-4535-908f-be2ff60927d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:39.534207 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82a52c-ba75-4535-908f-be2ff60927d2 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:39.545918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1275/2545] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:09:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:43.776258 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=49 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:44.265650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c7c78-38c4-477f-882f-1654722553ca service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:44.275365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c7c78-38c4-477f-882f-1654722553ca service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:44.280208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f36536-c746-4eca-93e6-e5e550565fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:44.280778 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f36536-c746-4eca-93e6-e5e550565fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:44.281057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f36536-c746-4eca-93e6-e5e550565fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:44.281326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f36536-c746-4eca-93e6-e5e550565fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:44.283231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-78f36536-c746-4eca-93e6-e5e550565fd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:44.298308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f36536-c746-4eca-93e6-e5e550565fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:44.299842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f36536-c746-4eca-93e6-e5e550565fd0 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:44.312146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1271/2546] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:09:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:45.025496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee739f5-b589-4a47-ad66-f1082076dac1 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:45.026002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee739f5-b589-4a47-ad66-f1082076dac1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:45.026107 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee739f5-b589-4a47-ad66-f1082076dac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:45.045308 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-8ee739f5-b589-4a47-ad66-f1082076dac1 None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:45.046891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-8ee739f5-b589-4a47-ad66-f1082076dac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:45.113439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1276/2547] 10.222.0.121 () {58 vars in 972 bytes} [Thu May 26 18:09:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:45.130227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f36536-c746-4eca-93e6-e5e550565fd0 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:45.207001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f36536-c746-4eca-93e6-e5e550565fd0 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:45.212043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704cbaa-4dff-484b-a014-c708395e378b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:45.212687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704cbaa-4dff-484b-a014-c708395e378b service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:45.212964 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704cbaa-4dff-484b-a014-c708395e378b service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:45.213253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704cbaa-4dff-484b-a014-c708395e378b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:45.222371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704cbaa-4dff-484b-a014-c708395e378b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:45.223982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704cbaa-4dff-484b-a014-c708395e378b service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:45.233396 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1272/2548] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:09:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:09:55.108635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=78 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:55.229144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=38 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:00.878631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847743c-f83c-4fb5-9b7a-81f792c2cc3d None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:00.879182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847743c-f83c-4fb5-9b7a-81f792c2cc3d None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:00.879301 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847743c-f83c-4fb5-9b7a-81f792c2cc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:00.896967 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d847743c-f83c-4fb5-9b7a-81f792c2cc3d None None] MFA Rules not processed for user `e7e74d9bfa6144ae8ad7e3156fb42e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:00.898162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d847743c-f83c-4fb5-9b7a-81f792c2cc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:00.904975 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1277/2549] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:10:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:01.219457 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704cbaa-4dff-484b-a014-c708395e378b service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:01.228945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704cbaa-4dff-484b-a014-c708395e378b service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:01.232691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0dad5-4a06-4403-a905-512487c71fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:01.233285 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0dad5-4a06-4403-a905-512487c71fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:01.233565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0dad5-4a06-4403-a905-512487c71fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:01.233840 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0dad5-4a06-4403-a905-512487c71fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:01.236149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-12e0dad5-4a06-4403-a905-512487c71fb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:01.245856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0dad5-4a06-4403-a905-512487c71fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:01.247311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0dad5-4a06-4403-a905-512487c71fb9 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:01.255923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1273/2550] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:10:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:02.016138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847743c-f83c-4fb5-9b7a-81f792c2cc3d None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:02.016138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d847743c-f83c-4fb5-9b7a-81f792c2cc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:02.087706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847743c-f83c-4fb5-9b7a-81f792c2cc3d None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:02.092636 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42e89a-05dc-42f1-a872-948e45b05332 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:02.093293 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42e89a-05dc-42f1-a872-948e45b05332 service placement] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:02.093573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42e89a-05dc-42f1-a872-948e45b05332 service placement] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:02.094348 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42e89a-05dc-42f1-a872-948e45b05332 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:02.096260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3b42e89a-05dc-42f1-a872-948e45b05332 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:02.108430 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42e89a-05dc-42f1-a872-948e45b05332 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:02.110429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42e89a-05dc-42f1-a872-948e45b05332 service placement] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:02.127503 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1278/2551] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:10:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:05.373261 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0dad5-4a06-4403-a905-512487c71fb9 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:05.384089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0dad5-4a06-4403-a905-512487c71fb9 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:05.403806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e08b79-5d75-4e4f-ba22-297e393a3737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:05.405187 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e08b79-5d75-4e4f-ba22-297e393a3737 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:05.405479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e08b79-5d75-4e4f-ba22-297e393a3737 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:05.405756 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e08b79-5d75-4e4f-ba22-297e393a3737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:05.408741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e3e08b79-5d75-4e4f-ba22-297e393a3737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:05.426083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e08b79-5d75-4e4f-ba22-297e393a3737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:05.427842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e08b79-5d75-4e4f-ba22-297e393a3737 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:05.436398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1274/2552] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:10:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:10:12.116361 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=35 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:15.432710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=12 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:21.951031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42e89a-05dc-42f1-a872-948e45b05332 service placement] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:21.960353 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42e89a-05dc-42f1-a872-948e45b05332 service placement] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:21.964173 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe646c29-9b76-4145-a9a8-f8cf91f057f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:21.964812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe646c29-9b76-4145-a9a8-f8cf91f057f5 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:21.965090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe646c29-9b76-4145-a9a8-f8cf91f057f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:21.965359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe646c29-9b76-4145-a9a8-f8cf91f057f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:21.974704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe646c29-9b76-4145-a9a8-f8cf91f057f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:21.977339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe646c29-9b76-4145-a9a8-f8cf91f057f5 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:21.986568 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1279/2553] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:10:21 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:24.256529 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e08b79-5d75-4e4f-ba22-297e393a3737 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:24.258573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e3e08b79-5d75-4e4f-ba22-297e393a3737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:24.275274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e08b79-5d75-4e4f-ba22-297e393a3737 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:24.281085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91530ab-57bb-4ca8-95da-ced7eb39e826 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:24.281806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91530ab-57bb-4ca8-95da-ced7eb39e826 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:24.282150 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91530ab-57bb-4ca8-95da-ced7eb39e826 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:24.282517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91530ab-57bb-4ca8-95da-ced7eb39e826 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:24.284397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e91530ab-57bb-4ca8-95da-ced7eb39e826 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:24.298649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91530ab-57bb-4ca8-95da-ced7eb39e826 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:24.300318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91530ab-57bb-4ca8-95da-ced7eb39e826 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:24.312425 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1275/2554] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:10:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:31.982166 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=6 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:34.307522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:42.392147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe646c29-9b76-4145-a9a8-f8cf91f057f5 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:42.401773 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe646c29-9b76-4145-a9a8-f8cf91f057f5 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:42.405873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5d57a-d64e-4372-941d-4f3ac82b2da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:42.406556 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5d57a-d64e-4372-941d-4f3ac82b2da7 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:42.412682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5d57a-d64e-4372-941d-4f3ac82b2da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:42.412682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5d57a-d64e-4372-941d-4f3ac82b2da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:42.412682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5ef5d57a-d64e-4372-941d-4f3ac82b2da7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:42.475306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5d57a-d64e-4372-941d-4f3ac82b2da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:42.476734 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5d57a-d64e-4372-941d-4f3ac82b2da7 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:42.487380 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1280/2555] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:10:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:46.502946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91530ab-57bb-4ca8-95da-ced7eb39e826 service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:46.504709 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e91530ab-57bb-4ca8-95da-ced7eb39e826 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:46.570446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91530ab-57bb-4ca8-95da-ced7eb39e826 service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:46.575172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccec62-e9d6-48ed-9662-99a23bab0db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:46.576034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccec62-e9d6-48ed-9662-99a23bab0db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:46.576311 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccec62-e9d6-48ed-9662-99a23bab0db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:46.576604 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccec62-e9d6-48ed-9662-99a23bab0db0 admin admin] PATH_INFO: `/v3/users/4ffcb99923b346938eb72c0b2786e815` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:46.592001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccec62-e9d6-48ed-9662-99a23bab0db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ffcb99923b346938eb72c0b2786e815)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:46.594392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccec62-e9d6-48ed-9662-99a23bab0db0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:46.823045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-04ccec62-e9d6-48ed-9662-99a23bab0db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ffcb99923b346938eb72c0b2786e815'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:46.829966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-04ccec62-e9d6-48ed-9662-99a23bab0db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ffcb99923b346938eb72c0b2786e815'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:46.830675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-04ccec62-e9d6-48ed-9662-99a23bab0db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ffcb99923b346938eb72c0b2786e815'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:46.843205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-04ccec62-e9d6-48ed-9662-99a23bab0db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ffcb99923b346938eb72c0b2786e815'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:46.900277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-04ccec62-e9d6-48ed-9662-99a23bab0db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ffcb99923b346938eb72c0b2786e815'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:46.907081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1276/2556] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:10:46 2022] DELETE /identity/v3/users/4ffcb99923b346938eb72c0b2786e815 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:10:47.153885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5d57a-d64e-4372-941d-4f3ac82b2da7 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:47.195083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5d57a-d64e-4372-941d-4f3ac82b2da7 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:47.251307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:47.252263 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:47.252546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:47.252810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2 admin admin] PATH_INFO: `/v3/projects/69b110a601c541328917618364dcad5b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:47.263824 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b110a601c541328917618364dcad5b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:47.265743 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:47.403763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b110a601c541328917618364dcad5b', 'request_id': 'req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:47.500428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b110a601c541328917618364dcad5b', 'request_id': 'req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:47.507790 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b110a601c541328917618364dcad5b', 'request_id': 'req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:47.509760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:47.518846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2 admin admin] The token cache is being invalidate because project 69b110a601c541328917618364dcad5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:10:47.519143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:47.519766 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4f56b3a8-3d56-4b14-b0a2-7a033261a8b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:47.521286 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1281/2557] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:10:47 2022] DELETE /identity/v3/projects/69b110a601c541328917618364dcad5b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:10:47.526169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540adcb-74f2-45be-9aa2-9da4e2b382d0 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:47.526643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540adcb-74f2-45be-9aa2-9da4e2b382d0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:47.526805 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540adcb-74f2-45be-9aa2-9da4e2b382d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:47.554461 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b540adcb-74f2-45be-9aa2-9da4e2b382d0 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:47.555709 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b540adcb-74f2-45be-9aa2-9da4e2b382d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:47.614796 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1277/2558] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:47.623635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb3eaf-1edb-4e06-a36c-8e1964e50406 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:47.623920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb3eaf-1edb-4e06-a36c-8e1964e50406 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:47.624302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb3eaf-1edb-4e06-a36c-8e1964e50406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:47.643299 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a1eb3eaf-1edb-4e06-a36c-8e1964e50406 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:47.644484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a1eb3eaf-1edb-4e06-a36c-8e1964e50406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:47.652247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1282/2559] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:47.656760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540adcb-74f2-45be-9aa2-9da4e2b382d0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:47.666330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540adcb-74f2-45be-9aa2-9da4e2b382d0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:47.671118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d0c57-5024-4fc6-be9f-0da787efa417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:47.671780 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d0c57-5024-4fc6-be9f-0da787efa417 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:47.672066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d0c57-5024-4fc6-be9f-0da787efa417 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:47.672341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d0c57-5024-4fc6-be9f-0da787efa417 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:47.673091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d0c57-5024-4fc6-be9f-0da787efa417 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:10:47.673395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d0c57-5024-4fc6-be9f-0da787efa417 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:47.675068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d0c57-5024-4fc6-be9f-0da787efa417 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:47.682269 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:47.688072 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1278/2560] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:10:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:10:47.692570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb3eaf-1edb-4e06-a36c-8e1964e50406 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:47.703458 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb3eaf-1edb-4e06-a36c-8e1964e50406 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:47.708238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbdde1-0df8-4d33-98b3-f6c9cb98d060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:47.708890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbdde1-0df8-4d33-98b3-f6c9cb98d060 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:47.709176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbdde1-0df8-4d33-98b3-f6c9cb98d060 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:47.709453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbdde1-0df8-4d33-98b3-f6c9cb98d060 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:47.710165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbdde1-0df8-4d33-98b3-f6c9cb98d060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:47.712142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbdde1-0df8-4d33-98b3-f6c9cb98d060 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:47.852881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1283/2561] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:10:47 2022] POST /identity/v3/projects => generated 374 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:10:47.856796 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d0c57-5024-4fc6-be9f-0da787efa417 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:47.866657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d0c57-5024-4fc6-be9f-0da787efa417 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:47.922401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb032735-9091-45b2-910f-8ea9586c5c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:47.923104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032735-9091-45b2-910f-8ea9586c5c3b admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:47.923384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032735-9091-45b2-910f-8ea9586c5c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:47.923654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032735-9091-45b2-910f-8ea9586c5c3b admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:47.924345 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb032735-9091-45b2-910f-8ea9586c5c3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:47.926244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb032735-9091-45b2-910f-8ea9586c5c3b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:48.015728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1279/2562] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:10:47 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:10:48.019829 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbdde1-0df8-4d33-98b3-f6c9cb98d060 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:48.031059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbdde1-0df8-4d33-98b3-f6c9cb98d060 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:48.035787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61818acc-9b88-40bf-8f92-657abafc533d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:48.036531 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61818acc-9b88-40bf-8f92-657abafc533d admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.036859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61818acc-9b88-40bf-8f92-657abafc533d admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.037145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61818acc-9b88-40bf-8f92-657abafc533d admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.037872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61818acc-9b88-40bf-8f92-657abafc533d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:48.039776 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61818acc-9b88-40bf-8f92-657abafc533d admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:48.051098 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1284/2563] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:10:48 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:10:48.055353 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb032735-9091-45b2-910f-8ea9586c5c3b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:48.065595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb032735-9091-45b2-910f-8ea9586c5c3b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:48.070192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d6bc2-b866-4188-bf2c-3260a497790d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:48.070899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d6bc2-b866-4188-bf2c-3260a497790d admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.071149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d6bc2-b866-4188-bf2c-3260a497790d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.071423 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d6bc2-b866-4188-bf2c-3260a497790d admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.072134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d6bc2-b866-4188-bf2c-3260a497790d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:48.073978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d6bc2-b866-4188-bf2c-3260a497790d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:48.084434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1280/2564] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:10:48 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:10:48.088736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61818acc-9b88-40bf-8f92-657abafc533d admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:48.099676 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61818acc-9b88-40bf-8f92-657abafc533d admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:48.104357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9683770-ba0c-41cb-97c5-f0e04848281f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:48.105193 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9683770-ba0c-41cb-97c5-f0e04848281f admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.105555 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9683770-ba0c-41cb-97c5-f0e04848281f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.105860 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9683770-ba0c-41cb-97c5-f0e04848281f admin admin] PATH_INFO: `/v3/projects/206c0f96312d4ed19afad6e5823a1d81/users/99da2cb97d814bdeb288a3e102e8fd71/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.120699 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9683770-ba0c-41cb-97c5-f0e04848281f admin admin] RBAC: Authorizing `identity:create_grant(project_id=206c0f96312d4ed19afad6e5823a1d81, user_id=99da2cb97d814bdeb288a3e102e8fd71, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:48.123696 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9683770-ba0c-41cb-97c5-f0e04848281f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:48.203780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1285/2565] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:10:48 2022] PUT /identity/v3/projects/206c0f96312d4ed19afad6e5823a1d81/users/99da2cb97d814bdeb288a3e102e8fd71/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:10:48.213965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b27ad6-0f3f-469e-b1b3-30d22ceecd13 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.224966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b27ad6-0f3f-469e-b1b3-30d22ceecd13 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.224966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b27ad6-0f3f-469e-b1b3-30d22ceecd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.243122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f0b27ad6-0f3f-469e-b1b3-30d22ceecd13 None None] MFA Rules not processed for user `99da2cb97d814bdeb288a3e102e8fd71`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:48.244404 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f0b27ad6-0f3f-469e-b1b3-30d22ceecd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:48.308386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1281/2566] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:10:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:48.313027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9683770-ba0c-41cb-97c5-f0e04848281f admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:48.374069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9683770-ba0c-41cb-97c5-f0e04848281f admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:48.378681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09d965-0954-44c5-99db-63eb3f2e2181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:48.379368 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09d965-0954-44c5-99db-63eb3f2e2181 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.379650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09d965-0954-44c5-99db-63eb3f2e2181 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.379951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09d965-0954-44c5-99db-63eb3f2e2181 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.380645 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09d965-0954-44c5-99db-63eb3f2e2181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:48.382550 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09d965-0954-44c5-99db-63eb3f2e2181 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:48.490463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1286/2567] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:10:48 2022] POST /identity/v3/projects => generated 370 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:10:48.494400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b27ad6-0f3f-469e-b1b3-30d22ceecd13 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:48.506245 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b27ad6-0f3f-469e-b1b3-30d22ceecd13 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:48.567952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f747dc-b3e6-48aa-bd33-e96f12477989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:48.568681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f747dc-b3e6-48aa-bd33-e96f12477989 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.568901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f747dc-b3e6-48aa-bd33-e96f12477989 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.569172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f747dc-b3e6-48aa-bd33-e96f12477989 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.569871 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f747dc-b3e6-48aa-bd33-e96f12477989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:48.571847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f747dc-b3e6-48aa-bd33-e96f12477989 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:48.582620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:48.667009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1282/2568] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:10:48 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:10:48.670731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09d965-0954-44c5-99db-63eb3f2e2181 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:48.682003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09d965-0954-44c5-99db-63eb3f2e2181 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:48.687960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c59bf4-41b0-4a49-b0b5-28586b50b67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:48.689161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c59bf4-41b0-4a49-b0b5-28586b50b67b admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.689161 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c59bf4-41b0-4a49-b0b5-28586b50b67b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.689435 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c59bf4-41b0-4a49-b0b5-28586b50b67b admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.689945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c59bf4-41b0-4a49-b0b5-28586b50b67b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:48.691601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c59bf4-41b0-4a49-b0b5-28586b50b67b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:48.704135 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1287/2569] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:10:48 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:10:48.708926 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f747dc-b3e6-48aa-bd33-e96f12477989 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:48.720999 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f747dc-b3e6-48aa-bd33-e96f12477989 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:48.725528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13031351-b686-4a17-9b26-c56e7eeb4510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:48.726571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13031351-b686-4a17-9b26-c56e7eeb4510 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.726879 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13031351-b686-4a17-9b26-c56e7eeb4510 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.727153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13031351-b686-4a17-9b26-c56e7eeb4510 admin admin] PATH_INFO: `/v3/projects/5fcd46c3a30347e6aabd6b17fddb0487/users/556b84231ed248e6a2a8c8477eade2a9/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.742471 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13031351-b686-4a17-9b26-c56e7eeb4510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fcd46c3a30347e6aabd6b17fddb0487, user_id=556b84231ed248e6a2a8c8477eade2a9, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:48.745191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13031351-b686-4a17-9b26-c56e7eeb4510 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:48.797945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1283/2570] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:10:48 2022] PUT /identity/v3/projects/5fcd46c3a30347e6aabd6b17fddb0487/users/556b84231ed248e6a2a8c8477eade2a9/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:10:48.802115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c59bf4-41b0-4a49-b0b5-28586b50b67b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:48.813296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c59bf4-41b0-4a49-b0b5-28586b50b67b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:48.870279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160c130-a3a9-40de-b133-7a16d15b25b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:48.871101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160c130-a3a9-40de-b133-7a16d15b25b1 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.871977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160c130-a3a9-40de-b133-7a16d15b25b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.872252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160c130-a3a9-40de-b133-7a16d15b25b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.872973 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160c130-a3a9-40de-b133-7a16d15b25b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:48.874557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160c130-a3a9-40de-b133-7a16d15b25b1 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:48.876573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:48.884486 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1288/2571] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:10:48 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:10:48.888810 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13031351-b686-4a17-9b26-c56e7eeb4510 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:48.899019 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13031351-b686-4a17-9b26-c56e7eeb4510 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:48.903810 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a26ec-eb94-4a74-b116-7665444d174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:48.904714 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a26ec-eb94-4a74-b116-7665444d174c admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.904991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a26ec-eb94-4a74-b116-7665444d174c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.905263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a26ec-eb94-4a74-b116-7665444d174c admin admin] PATH_INFO: `/v3/projects/5fcd46c3a30347e6aabd6b17fddb0487/users/556b84231ed248e6a2a8c8477eade2a9/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.907248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a26ec-eb94-4a74-b116-7665444d174c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fcd46c3a30347e6aabd6b17fddb0487, user_id=556b84231ed248e6a2a8c8477eade2a9, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:48.909865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a26ec-eb94-4a74-b116-7665444d174c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:48.920799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1284/2572] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:10:48 2022] PUT /identity/v3/projects/5fcd46c3a30347e6aabd6b17fddb0487/users/556b84231ed248e6a2a8c8477eade2a9/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:10:48.932394 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4a6a3-a8b2-4978-b98b-e8c9bd1f9722 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:48.932683 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4a6a3-a8b2-4978-b98b-e8c9bd1f9722 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:48.933009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4a6a3-a8b2-4978-b98b-e8c9bd1f9722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:48.959967 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-51d4a6a3-a8b2-4978-b98b-e8c9bd1f9722 None None] MFA Rules not processed for user `556b84231ed248e6a2a8c8477eade2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:48.961241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-51d4a6a3-a8b2-4978-b98b-e8c9bd1f9722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:49.029328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1289/2573] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:10:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:49.040641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a26ec-eb94-4a74-b116-7665444d174c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:49.042402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-643a26ec-eb94-4a74-b116-7665444d174c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:49.127746 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a26ec-eb94-4a74-b116-7665444d174c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:49.131438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda8847-5e71-48ca-a910-c54049711848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:49.132115 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda8847-5e71-48ca-a910-c54049711848 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:49.132397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda8847-5e71-48ca-a910-c54049711848 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:49.132672 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda8847-5e71-48ca-a910-c54049711848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:49.142027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda8847-5e71-48ca-a910-c54049711848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:49.143488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda8847-5e71-48ca-a910-c54049711848 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:49.154937 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1285/2574] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:10:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:49.381533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4a6a3-a8b2-4978-b98b-e8c9bd1f9722 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:49.393114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4a6a3-a8b2-4978-b98b-e8c9bd1f9722 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:49.397041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd14ca-ddcd-4fae-aab8-ad53bda34666 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:49.397686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd14ca-ddcd-4fae-aab8-ad53bda34666 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:49.397966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd14ca-ddcd-4fae-aab8-ad53bda34666 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:49.398237 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd14ca-ddcd-4fae-aab8-ad53bda34666 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:49.456154 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd14ca-ddcd-4fae-aab8-ad53bda34666 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:49.457778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd14ca-ddcd-4fae-aab8-ad53bda34666 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:49.467075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1290/2575] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:10:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:55.713796 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda8847-5e71-48ca-a910-c54049711848 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:55.725508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda8847-5e71-48ca-a910-c54049711848 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:55.729202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d0f5a-7638-46b9-aebc-5cf9e11c8170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:55.729841 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d0f5a-7638-46b9-aebc-5cf9e11c8170 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:55.730123 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d0f5a-7638-46b9-aebc-5cf9e11c8170 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:55.730430 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d0f5a-7638-46b9-aebc-5cf9e11c8170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:55.732523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-143d0f5a-7638-46b9-aebc-5cf9e11c8170 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:55.816540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d0f5a-7638-46b9-aebc-5cf9e11c8170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:55.817979 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d0f5a-7638-46b9-aebc-5cf9e11c8170 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:55.836402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1286/2576] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:10:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:59.462542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=63 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:04.566609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd14ca-ddcd-4fae-aab8-ad53bda34666 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:04.567964 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-98bd14ca-ddcd-4fae-aab8-ad53bda34666 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:04.648509 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd14ca-ddcd-4fae-aab8-ad53bda34666 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:04.652459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ff7a9-39b5-4b93-91b6-7d6116a8bf2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:04.653167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ff7a9-39b5-4b93-91b6-7d6116a8bf2e service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:04.653444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ff7a9-39b5-4b93-91b6-7d6116a8bf2e service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:04.653715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ff7a9-39b5-4b93-91b6-7d6116a8bf2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:04.655576 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-289ff7a9-39b5-4b93-91b6-7d6116a8bf2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:04.730758 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ff7a9-39b5-4b93-91b6-7d6116a8bf2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:04.732241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ff7a9-39b5-4b93-91b6-7d6116a8bf2e service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:04.752845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1291/2577] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:05.833766 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:08.491301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d0f5a-7638-46b9-aebc-5cf9e11c8170 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:08.493149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-143d0f5a-7638-46b9-aebc-5cf9e11c8170 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:08.574673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d0f5a-7638-46b9-aebc-5cf9e11c8170 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:08.580779 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d1c4b-48be-49b8-8b41-8071d36ca683 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:08.581559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d1c4b-48be-49b8-8b41-8071d36ca683 service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:08.581700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d1c4b-48be-49b8-8b41-8071d36ca683 service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:08.581984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d1c4b-48be-49b8-8b41-8071d36ca683 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:08.584294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-179d1c4b-48be-49b8-8b41-8071d36ca683 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:08.596251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d1c4b-48be-49b8-8b41-8071d36ca683 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:08.597933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d1c4b-48be-49b8-8b41-8071d36ca683 service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:08.609650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1287/2578] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:11:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:08.644760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ff7a9-39b5-4b93-91b6-7d6116a8bf2e service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:08.647097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-289ff7a9-39b5-4b93-91b6-7d6116a8bf2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:08.663068 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ff7a9-39b5-4b93-91b6-7d6116a8bf2e service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:08.667894 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b79509-df7a-4a9b-b796-a9f35c3638e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:08.668771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b79509-df7a-4a9b-b796-a9f35c3638e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:08.669056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b79509-df7a-4a9b-b796-a9f35c3638e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:08.669333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b79509-df7a-4a9b-b796-a9f35c3638e0 admin admin] PATH_INFO: `/v3/users/fe1c86c411834f1c8d2775a5b2c5f1d1` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:08.670583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b79509-df7a-4a9b-b796-a9f35c3638e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe1c86c411834f1c8d2775a5b2c5f1d1)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:08.672538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b79509-df7a-4a9b-b796-a9f35c3638e0 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:08.857887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-45b79509-df7a-4a9b-b796-a9f35c3638e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe1c86c411834f1c8d2775a5b2c5f1d1'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:08.865721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-45b79509-df7a-4a9b-b796-a9f35c3638e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe1c86c411834f1c8d2775a5b2c5f1d1'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:08.866418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-45b79509-df7a-4a9b-b796-a9f35c3638e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe1c86c411834f1c8d2775a5b2c5f1d1'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:08.875079 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:08.879643 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-45b79509-df7a-4a9b-b796-a9f35c3638e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe1c86c411834f1c8d2775a5b2c5f1d1'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:08.951396 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-45b79509-df7a-4a9b-b796-a9f35c3638e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe1c86c411834f1c8d2775a5b2c5f1d1'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:08.958510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1292/2579] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:11:08 2022] DELETE /identity/v3/users/fe1c86c411834f1c8d2775a5b2c5f1d1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:09.263240 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d1c4b-48be-49b8-8b41-8071d36ca683 service placement] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:09.276320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d1c4b-48be-49b8-8b41-8071d36ca683 service placement] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:09.340922 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73ed58-a1bc-473f-a027-2b5d5230fa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:09.341724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73ed58-a1bc-473f-a027-2b5d5230fa22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:09.342002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73ed58-a1bc-473f-a027-2b5d5230fa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:09.342274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73ed58-a1bc-473f-a027-2b5d5230fa22 admin admin] PATH_INFO: `/v3/projects/68bd0110e4fd4e6dbef9f65397908292` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:09.343547 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73ed58-a1bc-473f-a027-2b5d5230fa22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68bd0110e4fd4e6dbef9f65397908292)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:09.345485 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73ed58-a1bc-473f-a027-2b5d5230fa22 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:09.466096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1d73ed58-a1bc-473f-a027-2b5d5230fa22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68bd0110e4fd4e6dbef9f65397908292', 'request_id': 'req-1d73ed58-a1bc-473f-a027-2b5d5230fa22'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:09.534179 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1d73ed58-a1bc-473f-a027-2b5d5230fa22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68bd0110e4fd4e6dbef9f65397908292', 'request_id': 'req-1d73ed58-a1bc-473f-a027-2b5d5230fa22'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:09.541087 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1d73ed58-a1bc-473f-a027-2b5d5230fa22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68bd0110e4fd4e6dbef9f65397908292', 'request_id': 'req-1d73ed58-a1bc-473f-a027-2b5d5230fa22'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:09.553201 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1d73ed58-a1bc-473f-a027-2b5d5230fa22 admin admin] The token cache is being invalidate because project 68bd0110e4fd4e6dbef9f65397908292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:11:09.553502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1d73ed58-a1bc-473f-a027-2b5d5230fa22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:09.554139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1d73ed58-a1bc-473f-a027-2b5d5230fa22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:09.555735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1288/2580] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:11:09 2022] DELETE /identity/v3/projects/68bd0110e4fd4e6dbef9f65397908292 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:09.561586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922b2e4-908d-4aa1-85ec-d71dfd4575f6 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:09.561974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922b2e4-908d-4aa1-85ec-d71dfd4575f6 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:09.562252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922b2e4-908d-4aa1-85ec-d71dfd4575f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:09.612720 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-6922b2e4-908d-4aa1-85ec-d71dfd4575f6 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:09.613831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-6922b2e4-908d-4aa1-85ec-d71dfd4575f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:09.674238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1293/2581] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:09.683420 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb797ebe-ca71-4f7f-8ae7-26070fd1d219 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:09.683797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb797ebe-ca71-4f7f-8ae7-26070fd1d219 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:09.684080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb797ebe-ca71-4f7f-8ae7-26070fd1d219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:09.702811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bb797ebe-ca71-4f7f-8ae7-26070fd1d219 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:09.704034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bb797ebe-ca71-4f7f-8ae7-26070fd1d219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:09.711865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1289/2582] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:11:09.717070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922b2e4-908d-4aa1-85ec-d71dfd4575f6 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:09.726805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922b2e4-908d-4aa1-85ec-d71dfd4575f6 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:09.732538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6351839-cdb7-426e-98b9-ae99f277abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:09.733202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6351839-cdb7-426e-98b9-ae99f277abc3 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:09.733486 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6351839-cdb7-426e-98b9-ae99f277abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:09.733881 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6351839-cdb7-426e-98b9-ae99f277abc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:09.734487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6351839-cdb7-426e-98b9-ae99f277abc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:11:09.734803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6351839-cdb7-426e-98b9-ae99f277abc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:09.736431 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6351839-cdb7-426e-98b9-ae99f277abc3 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:09.748408 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1294/2583] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:11:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:11:09.752772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb797ebe-ca71-4f7f-8ae7-26070fd1d219 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:09.762591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb797ebe-ca71-4f7f-8ae7-26070fd1d219 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:09.767416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40af67-1b9e-4ebb-9d4c-0aca81bb38f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:09.768068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40af67-1b9e-4ebb-9d4c-0aca81bb38f9 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:09.768350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40af67-1b9e-4ebb-9d4c-0aca81bb38f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:09.768620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40af67-1b9e-4ebb-9d4c-0aca81bb38f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:09.769306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40af67-1b9e-4ebb-9d4c-0aca81bb38f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:09.771267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40af67-1b9e-4ebb-9d4c-0aca81bb38f9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:09.782592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:09.868614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1290/2584] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:11:09 2022] POST /identity/v3/projects => generated 378 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:09.873474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6351839-cdb7-426e-98b9-ae99f277abc3 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:09.886297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6351839-cdb7-426e-98b9-ae99f277abc3 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:09.948236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a7080-29bb-4655-ae48-dc1baf064d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:09.948236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a7080-29bb-4655-ae48-dc1baf064d1e admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:09.948236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a7080-29bb-4655-ae48-dc1baf064d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:09.948236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a7080-29bb-4655-ae48-dc1baf064d1e admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:09.949123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a7080-29bb-4655-ae48-dc1baf064d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:09.951252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a7080-29bb-4655-ae48-dc1baf064d1e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:10.033795 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1295/2585] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:11:09 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:10.037896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40af67-1b9e-4ebb-9d4c-0aca81bb38f9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:10.048982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40af67-1b9e-4ebb-9d4c-0aca81bb38f9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:10.053988 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b709a22-2a7f-4e32-911b-60d508c6060e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:10.058795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b709a22-2a7f-4e32-911b-60d508c6060e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:10.058795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b709a22-2a7f-4e32-911b-60d508c6060e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:10.058795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b709a22-2a7f-4e32-911b-60d508c6060e admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:10.058795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b709a22-2a7f-4e32-911b-60d508c6060e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:10.058795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b709a22-2a7f-4e32-911b-60d508c6060e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:10.069177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1291/2586] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:11:10 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:10.073261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a7080-29bb-4655-ae48-dc1baf064d1e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:10.083917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a7080-29bb-4655-ae48-dc1baf064d1e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:10.088571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3009e-e160-42ef-b89b-75e23d19b64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:10.089257 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3009e-e160-42ef-b89b-75e23d19b64f admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:10.089535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3009e-e160-42ef-b89b-75e23d19b64f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:10.089808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3009e-e160-42ef-b89b-75e23d19b64f admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:10.090526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3009e-e160-42ef-b89b-75e23d19b64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:10.094846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3009e-e160-42ef-b89b-75e23d19b64f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:10.103725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1296/2587] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:11:10 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:10.108511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b709a22-2a7f-4e32-911b-60d508c6060e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:10.119148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b709a22-2a7f-4e32-911b-60d508c6060e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:10.125157 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e098d6-68e2-45e5-b6a9-6d8e2cfc7b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:10.126043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e098d6-68e2-45e5-b6a9-6d8e2cfc7b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:10.126323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e098d6-68e2-45e5-b6a9-6d8e2cfc7b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:10.126692 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e098d6-68e2-45e5-b6a9-6d8e2cfc7b7e admin admin] PATH_INFO: `/v3/projects/d0c5ea7bbd294bd7b7729aa26b6c715c/users/7df2a24143654445bcb96df81e960c9d/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:10.142385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e098d6-68e2-45e5-b6a9-6d8e2cfc7b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c5ea7bbd294bd7b7729aa26b6c715c, user_id=7df2a24143654445bcb96df81e960c9d, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:10.145178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e098d6-68e2-45e5-b6a9-6d8e2cfc7b7e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:10.238130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1292/2588] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:11:10 2022] PUT /identity/v3/projects/d0c5ea7bbd294bd7b7729aa26b6c715c/users/7df2a24143654445bcb96df81e960c9d/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:10.243304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece1920-c0f1-486b-9c97-95695494366a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:10.243591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece1920-c0f1-486b-9c97-95695494366a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:10.243866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece1920-c0f1-486b-9c97-95695494366a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:10.261930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-7ece1920-c0f1-486b-9c97-95695494366a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:10.263247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7ece1920-c0f1-486b-9c97-95695494366a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:10.284586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:10.335976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1297/2589] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:10.346075 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e098d6-68e2-45e5-b6a9-6d8e2cfc7b7e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:10.347814 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c8e098d6-68e2-45e5-b6a9-6d8e2cfc7b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:10.407623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e098d6-68e2-45e5-b6a9-6d8e2cfc7b7e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:10.411945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b59c1-41f9-474d-aa47-853bc38f378f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:10.412602 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b59c1-41f9-474d-aa47-853bc38f378f service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:10.412939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b59c1-41f9-474d-aa47-853bc38f378f service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:10.413215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b59c1-41f9-474d-aa47-853bc38f378f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:10.423290 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b59c1-41f9-474d-aa47-853bc38f378f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:10.424808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b59c1-41f9-474d-aa47-853bc38f378f service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:10.435280 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1293/2590] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:15.456434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac47ab-57a3-4ed7-8b97-a684c8c61a7d None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:15.457053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac47ab-57a3-4ed7-8b97-a684c8c61a7d None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:15.457053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac47ab-57a3-4ed7-8b97-a684c8c61a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:15.488039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-59ac47ab-57a3-4ed7-8b97-a684c8c61a7d None None] MFA Rules not processed for user `7df2a24143654445bcb96df81e960c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:15.489275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-59ac47ab-57a3-4ed7-8b97-a684c8c61a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:15.565401 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1298/2591] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:11:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:15.594121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b59c1-41f9-474d-aa47-853bc38f378f service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:15.594121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-648b59c1-41f9-474d-aa47-853bc38f378f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:15.668023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b59c1-41f9-474d-aa47-853bc38f378f service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:15.673870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c87dcd-491c-4a92-9afb-d9bc90ca2e9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:15.674542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c87dcd-491c-4a92-9afb-d9bc90ca2e9b service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:15.674853 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c87dcd-491c-4a92-9afb-d9bc90ca2e9b service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:15.675130 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c87dcd-491c-4a92-9afb-d9bc90ca2e9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:15.684986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c87dcd-491c-4a92-9afb-d9bc90ca2e9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:15.686720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c87dcd-491c-4a92-9afb-d9bc90ca2e9b service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:15.697649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1294/2592] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:11:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:18.147742 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680cd19-f0fa-4139-9d54-4ba4cb218052 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.149375 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680cd19-f0fa-4139-9d54-4ba4cb218052 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.149375 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680cd19-f0fa-4139-9d54-4ba4cb218052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.169508 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1680cd19-f0fa-4139-9d54-4ba4cb218052 None None] MFA Rules not processed for user `7df2a24143654445bcb96df81e960c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:18.170955 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1680cd19-f0fa-4139-9d54-4ba4cb218052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:18.179391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1299/2593] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:11:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:18.598475 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c87dcd-491c-4a92-9afb-d9bc90ca2e9b service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.609136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c87dcd-491c-4a92-9afb-d9bc90ca2e9b service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.617384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e86e01-9054-4bec-b6bb-cdb26b69af48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.618123 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e86e01-9054-4bec-b6bb-cdb26b69af48 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.618428 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e86e01-9054-4bec-b6bb-cdb26b69af48 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.618939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e86e01-9054-4bec-b6bb-cdb26b69af48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.621825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-25e86e01-9054-4bec-b6bb-cdb26b69af48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:18.633523 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e86e01-9054-4bec-b6bb-cdb26b69af48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.635013 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e86e01-9054-4bec-b6bb-cdb26b69af48 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.645983 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1295/2594] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:11:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:11:28.163578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=50 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:28.641134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:33.645938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680cd19-f0fa-4139-9d54-4ba4cb218052 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:33.677729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680cd19-f0fa-4139-9d54-4ba4cb218052 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:33.684083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfdc97-4f0f-462a-b53c-64cf20cc74a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:33.684761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfdc97-4f0f-462a-b53c-64cf20cc74a8 service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:33.685041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfdc97-4f0f-462a-b53c-64cf20cc74a8 service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:33.685319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfdc97-4f0f-462a-b53c-64cf20cc74a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:33.687274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-41dfdc97-4f0f-462a-b53c-64cf20cc74a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:33.714203 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfdc97-4f0f-462a-b53c-64cf20cc74a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:33.715958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfdc97-4f0f-462a-b53c-64cf20cc74a8 service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:33.727054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1300/2595] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:11:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:43.498076 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e86e01-9054-4bec-b6bb-cdb26b69af48 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:43.499639 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-25e86e01-9054-4bec-b6bb-cdb26b69af48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:43.514896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e86e01-9054-4bec-b6bb-cdb26b69af48 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:43.519487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:43.520347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:43.520625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:43.520947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] PATH_INFO: `/v3/users/99da2cb97d814bdeb288a3e102e8fd71` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:43.522133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] RBAC: Authorizing `identity:delete_user(user_id=99da2cb97d814bdeb288a3e102e8fd71)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:43.524081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:43.722300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=18 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:43.776866 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99da2cb97d814bdeb288a3e102e8fd71'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:43.784253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99da2cb97d814bdeb288a3e102e8fd71'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:43.784941 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99da2cb97d814bdeb288a3e102e8fd71'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:43.797266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99da2cb97d814bdeb288a3e102e8fd71'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:43.859605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99da2cb97d814bdeb288a3e102e8fd71'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:43.865838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1296/2596] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:11:43 2022] DELETE /identity/v3/users/99da2cb97d814bdeb288a3e102e8fd71 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:43.870939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea417b-6cee-4ab6-af5c-22f16be97e89 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:43.871253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea417b-6cee-4ab6-af5c-22f16be97e89 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:43.871529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea417b-6cee-4ab6-af5c-22f16be97e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:43.889331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-0bea417b-6cee-4ab6-af5c-22f16be97e89 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:43.890526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0bea417b-6cee-4ab6-af5c-22f16be97e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:43.949483 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1301/2597] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:11:43.959095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:43.960482 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:44.017269 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0634b1-adb9-4922-8044-e20b6106521b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:44.020802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c2525-1a62-40a7-a818-651bd7e78ea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:44.021448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c2525-1a62-40a7-a818-651bd7e78ea0 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:44.021730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c2525-1a62-40a7-a818-651bd7e78ea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:44.022005 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c2525-1a62-40a7-a818-651bd7e78ea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:44.031094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c2525-1a62-40a7-a818-651bd7e78ea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:44.032513 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c2525-1a62-40a7-a818-651bd7e78ea0 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:44.042330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1297/2598] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:11:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:44.438673 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea417b-6cee-4ab6-af5c-22f16be97e89 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:44.440143 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0bea417b-6cee-4ab6-af5c-22f16be97e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:44.452718 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea417b-6cee-4ab6-af5c-22f16be97e89 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:44.457261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:44.458126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:44.458407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:44.458678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] PATH_INFO: `/v3/projects/206c0f96312d4ed19afad6e5823a1d81` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:44.459896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=206c0f96312d4ed19afad6e5823a1d81)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:44.461870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:44.603411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '206c0f96312d4ed19afad6e5823a1d81', 'request_id': 'req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:44.671090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '206c0f96312d4ed19afad6e5823a1d81', 'request_id': 'req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:44.677898 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '206c0f96312d4ed19afad6e5823a1d81', 'request_id': 'req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:44.688878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] The token cache is being invalidate because project 206c0f96312d4ed19afad6e5823a1d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:11:44.689168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:44.691849 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:44.691849 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1302/2599] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:11:44 2022] DELETE /identity/v3/projects/206c0f96312d4ed19afad6e5823a1d81 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:44.695508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c2525-1a62-40a7-a818-651bd7e78ea0 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:44.706150 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c2525-1a62-40a7-a818-651bd7e78ea0 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:44.764990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cc062-8282-4c49-b4a0-edbaca6973a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:44.766973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cc062-8282-4c49-b4a0-edbaca6973a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:44.766973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cc062-8282-4c49-b4a0-edbaca6973a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:44.766973 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cc062-8282-4c49-b4a0-edbaca6973a0 admin admin] PATH_INFO: `/v3/users/556b84231ed248e6a2a8c8477eade2a9` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:44.768223 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cc062-8282-4c49-b4a0-edbaca6973a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=556b84231ed248e6a2a8c8477eade2a9)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:44.770192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cc062-8282-4c49-b4a0-edbaca6973a0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:44.788652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:44.971226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ee5cc062-8282-4c49-b4a0-edbaca6973a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '556b84231ed248e6a2a8c8477eade2a9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:44.978159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ee5cc062-8282-4c49-b4a0-edbaca6973a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '556b84231ed248e6a2a8c8477eade2a9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:44.978887 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ee5cc062-8282-4c49-b4a0-edbaca6973a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '556b84231ed248e6a2a8c8477eade2a9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:44.990693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ee5cc062-8282-4c49-b4a0-edbaca6973a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '556b84231ed248e6a2a8c8477eade2a9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:45.048297 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ee5cc062-8282-4c49-b4a0-edbaca6973a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '556b84231ed248e6a2a8c8477eade2a9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:45.054890 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1298/2600] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:11:44 2022] DELETE /identity/v3/users/556b84231ed248e6a2a8c8477eade2a9 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:45.468453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:45.470424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:45.484174 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5bb56-99f5-4f82-a9f8-df93fe1f3d83 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:45.543089 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fd323-d641-4f62-ba0b-491598770c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:45.543967 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fd323-d641-4f62-ba0b-491598770c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:45.544252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fd323-d641-4f62-ba0b-491598770c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:45.544529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fd323-d641-4f62-ba0b-491598770c8f admin admin] PATH_INFO: `/v3/projects/5fcd46c3a30347e6aabd6b17fddb0487` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:45.545734 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fd323-d641-4f62-ba0b-491598770c8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fcd46c3a30347e6aabd6b17fddb0487)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:45.547709 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fd323-d641-4f62-ba0b-491598770c8f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:45.661712 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:45.678357 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1b6fd323-d641-4f62-ba0b-491598770c8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fcd46c3a30347e6aabd6b17fddb0487', 'request_id': 'req-1b6fd323-d641-4f62-ba0b-491598770c8f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:45.733726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1b6fd323-d641-4f62-ba0b-491598770c8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fcd46c3a30347e6aabd6b17fddb0487', 'request_id': 'req-1b6fd323-d641-4f62-ba0b-491598770c8f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:45.741658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1b6fd323-d641-4f62-ba0b-491598770c8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fcd46c3a30347e6aabd6b17fddb0487', 'request_id': 'req-1b6fd323-d641-4f62-ba0b-491598770c8f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:45.752931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1b6fd323-d641-4f62-ba0b-491598770c8f admin admin] The token cache is being invalidate because project 5fcd46c3a30347e6aabd6b17fddb0487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:11:45.753216 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1b6fd323-d641-4f62-ba0b-491598770c8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:45.753921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1b6fd323-d641-4f62-ba0b-491598770c8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:45.755446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1303/2601] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:11:45 2022] DELETE /identity/v3/projects/5fcd46c3a30347e6aabd6b17fddb0487 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:45.761253 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f30aaa-3fb5-471f-9f38-f1971d425439 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:45.761725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f30aaa-3fb5-471f-9f38-f1971d425439 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:45.761819 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f30aaa-3fb5-471f-9f38-f1971d425439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:45.780639 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-16f30aaa-3fb5-471f-9f38-f1971d425439 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:45.781835 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-16f30aaa-3fb5-471f-9f38-f1971d425439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:45.842127 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1299/2602] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:45.849928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6b312-c212-460a-b4e7-ba5bed940b55 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:45.850221 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6b312-c212-460a-b4e7-ba5bed940b55 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:45.850501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6b312-c212-460a-b4e7-ba5bed940b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:45.870298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-62e6b312-c212-460a-b4e7-ba5bed940b55 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:45.871530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-62e6b312-c212-460a-b4e7-ba5bed940b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:45.879364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1304/2603] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:45.883490 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f30aaa-3fb5-471f-9f38-f1971d425439 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:45.893189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f30aaa-3fb5-471f-9f38-f1971d425439 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:45.897700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1390ad-62c2-4799-bc4a-9d77dff75047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:45.898350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1390ad-62c2-4799-bc4a-9d77dff75047 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:45.898632 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1390ad-62c2-4799-bc4a-9d77dff75047 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:45.898950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1390ad-62c2-4799-bc4a-9d77dff75047 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:45.899663 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1390ad-62c2-4799-bc4a-9d77dff75047 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:11:45.899966 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1390ad-62c2-4799-bc4a-9d77dff75047 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:45.902241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1390ad-62c2-4799-bc4a-9d77dff75047 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:45.913902 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1300/2604] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:11:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:11:45.918737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6b312-c212-460a-b4e7-ba5bed940b55 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:45.927945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6b312-c212-460a-b4e7-ba5bed940b55 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:45.932855 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66351f8-a8f0-44fe-ae0a-a19f32aa5969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:45.933521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66351f8-a8f0-44fe-ae0a-a19f32aa5969 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:45.933805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66351f8-a8f0-44fe-ae0a-a19f32aa5969 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:45.934079 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66351f8-a8f0-44fe-ae0a-a19f32aa5969 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:45.934782 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66351f8-a8f0-44fe-ae0a-a19f32aa5969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:45.936707 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66351f8-a8f0-44fe-ae0a-a19f32aa5969 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:46.016077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1305/2605] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:11:45 2022] POST /identity/v3/projects => generated 388 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:46.019509 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1390ad-62c2-4799-bc4a-9d77dff75047 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:46.030374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1390ad-62c2-4799-bc4a-9d77dff75047 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:46.105331 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3506093-9d83-45ac-ba23-86d24ef562e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:46.106663 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3506093-9d83-45ac-ba23-86d24ef562e2 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:46.107286 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3506093-9d83-45ac-ba23-86d24ef562e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:46.107751 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3506093-9d83-45ac-ba23-86d24ef562e2 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:46.109006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3506093-9d83-45ac-ba23-86d24ef562e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:46.113838 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3506093-9d83-45ac-ba23-86d24ef562e2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:46.242957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1301/2606] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:11:46 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:46.247291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66351f8-a8f0-44fe-ae0a-a19f32aa5969 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:46.257731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66351f8-a8f0-44fe-ae0a-a19f32aa5969 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:46.262978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72797b-941d-4375-bc92-da0bd83a5699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:46.263653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72797b-941d-4375-bc92-da0bd83a5699 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:46.263936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72797b-941d-4375-bc92-da0bd83a5699 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:46.264212 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72797b-941d-4375-bc92-da0bd83a5699 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:46.264936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72797b-941d-4375-bc92-da0bd83a5699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:46.274603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72797b-941d-4375-bc92-da0bd83a5699 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:46.277177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1306/2607] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:11:46 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:46.281173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3506093-9d83-45ac-ba23-86d24ef562e2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:46.291739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3506093-9d83-45ac-ba23-86d24ef562e2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:46.296377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47643ac-08d7-4ffb-8639-2700d2ec2c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:46.297044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47643ac-08d7-4ffb-8639-2700d2ec2c7a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:46.297326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47643ac-08d7-4ffb-8639-2700d2ec2c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:46.297599 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47643ac-08d7-4ffb-8639-2700d2ec2c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:46.298561 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47643ac-08d7-4ffb-8639-2700d2ec2c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:46.299963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47643ac-08d7-4ffb-8639-2700d2ec2c7a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:46.312971 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:46.312971 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1302/2608] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:11:46 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:46.315884 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72797b-941d-4375-bc92-da0bd83a5699 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:46.333653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72797b-941d-4375-bc92-da0bd83a5699 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:46.341688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfcd4f-e131-46b4-9ab7-072652a27209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:46.341688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfcd4f-e131-46b4-9ab7-072652a27209 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:46.341688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfcd4f-e131-46b4-9ab7-072652a27209 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:46.341688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfcd4f-e131-46b4-9ab7-072652a27209 admin admin] PATH_INFO: `/v3/projects/7e5ee1f4885e4a54af253e7986a17540/users/3e14ea83f3ac4e5ca123382bfb8b5c9f/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:46.356191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfcd4f-e131-46b4-9ab7-072652a27209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5ee1f4885e4a54af253e7986a17540, user_id=3e14ea83f3ac4e5ca123382bfb8b5c9f, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:46.359037 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfcd4f-e131-46b4-9ab7-072652a27209 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:46.449895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1307/2609] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:11:46 2022] PUT /identity/v3/projects/7e5ee1f4885e4a54af253e7986a17540/users/3e14ea83f3ac4e5ca123382bfb8b5c9f/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:46.461409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87098955-5aaf-4402-ba38-a4121d7404f1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:46.461684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87098955-5aaf-4402-ba38-a4121d7404f1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:46.461960 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87098955-5aaf-4402-ba38-a4121d7404f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:46.495473 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-87098955-5aaf-4402-ba38-a4121d7404f1 None None] MFA Rules not processed for user `3e14ea83f3ac4e5ca123382bfb8b5c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:46.496707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-87098955-5aaf-4402-ba38-a4121d7404f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:46.566235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1303/2610] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:11:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:46.570845 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfcd4f-e131-46b4-9ab7-072652a27209 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:46.635680 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfcd4f-e131-46b4-9ab7-072652a27209 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:46.641506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf5800-4334-4657-bb7f-d5a7857fd755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:46.642195 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf5800-4334-4657-bb7f-d5a7857fd755 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:46.642480 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf5800-4334-4657-bb7f-d5a7857fd755 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:46.642751 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf5800-4334-4657-bb7f-d5a7857fd755 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:46.643472 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf5800-4334-4657-bb7f-d5a7857fd755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:46.645376 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf5800-4334-4657-bb7f-d5a7857fd755 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:46.767831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1308/2611] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:11:46 2022] POST /identity/v3/projects => generated 386 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:46.769391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87098955-5aaf-4402-ba38-a4121d7404f1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:46.781153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87098955-5aaf-4402-ba38-a4121d7404f1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:46.838860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab043b01-0502-466a-ab05-b58fa694383a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:46.839458 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab043b01-0502-466a-ab05-b58fa694383a admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:46.839738 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab043b01-0502-466a-ab05-b58fa694383a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:46.840007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab043b01-0502-466a-ab05-b58fa694383a admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:46.840718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab043b01-0502-466a-ab05-b58fa694383a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:46.842654 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab043b01-0502-466a-ab05-b58fa694383a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:46.963594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1304/2612] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:11:46 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:46.967516 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf5800-4334-4657-bb7f-d5a7857fd755 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:46.978060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf5800-4334-4657-bb7f-d5a7857fd755 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:46.984116 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5ac5a-0404-4106-af8b-b21d341d67ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:46.988745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5ac5a-0404-4106-af8b-b21d341d67ed admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:46.988745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5ac5a-0404-4106-af8b-b21d341d67ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:46.988745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5ac5a-0404-4106-af8b-b21d341d67ed admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:46.988745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5ac5a-0404-4106-af8b-b21d341d67ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:46.988745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5ac5a-0404-4106-af8b-b21d341d67ed admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:47.001314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1309/2613] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:11:46 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:47.005347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab043b01-0502-466a-ab05-b58fa694383a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:47.015865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab043b01-0502-466a-ab05-b58fa694383a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:47.020493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82a55d-e7cb-43e3-94e1-018102c76503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:47.023318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82a55d-e7cb-43e3-94e1-018102c76503 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:47.023318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82a55d-e7cb-43e3-94e1-018102c76503 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:47.023318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82a55d-e7cb-43e3-94e1-018102c76503 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:47.023318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82a55d-e7cb-43e3-94e1-018102c76503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:47.024007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82a55d-e7cb-43e3-94e1-018102c76503 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:47.034015 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1305/2614] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:11:47 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:47.038120 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5ac5a-0404-4106-af8b-b21d341d67ed admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:47.049310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5ac5a-0404-4106-af8b-b21d341d67ed admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:47.054349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b8c97-fae3-4210-b2e6-7a65de80a96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:47.055307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b8c97-fae3-4210-b2e6-7a65de80a96a admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:47.055589 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b8c97-fae3-4210-b2e6-7a65de80a96a admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:47.055865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b8c97-fae3-4210-b2e6-7a65de80a96a admin admin] PATH_INFO: `/v3/projects/4795844146324bb6a0c42c048481b52e/users/60c39215fb1a49bc85c652f78e36e279/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:47.071768 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b8c97-fae3-4210-b2e6-7a65de80a96a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4795844146324bb6a0c42c048481b52e, user_id=60c39215fb1a49bc85c652f78e36e279, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:47.074556 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b8c97-fae3-4210-b2e6-7a65de80a96a admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:47.146759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:47.152250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1310/2615] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:11:47 2022] PUT /identity/v3/projects/4795844146324bb6a0c42c048481b52e/users/60c39215fb1a49bc85c652f78e36e279/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:47.162598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934a94b-d9ca-4e30-8715-e5bcf448ffc1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:47.162923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934a94b-d9ca-4e30-8715-e5bcf448ffc1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:47.163207 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934a94b-d9ca-4e30-8715-e5bcf448ffc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:47.193611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-0934a94b-d9ca-4e30-8715-e5bcf448ffc1 None None] MFA Rules not processed for user `60c39215fb1a49bc85c652f78e36e279`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:47.194910 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0934a94b-d9ca-4e30-8715-e5bcf448ffc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:47.255251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:47.260845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1306/2616] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:11:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:47.265655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b8c97-fae3-4210-b2e6-7a65de80a96a admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:47.328056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b8c97-fae3-4210-b2e6-7a65de80a96a admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:47.332669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa658999-1448-437c-91a6-4ad4cd5227ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:47.333327 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa658999-1448-437c-91a6-4ad4cd5227ce admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:47.333607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa658999-1448-437c-91a6-4ad4cd5227ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:47.333879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa658999-1448-437c-91a6-4ad4cd5227ce admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:47.334569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa658999-1448-437c-91a6-4ad4cd5227ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:47.336485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa658999-1448-437c-91a6-4ad4cd5227ce admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:47.496497 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1311/2617] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:11:47 2022] POST /identity/v3/projects => generated 388 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:47.500596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934a94b-d9ca-4e30-8715-e5bcf448ffc1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:47.511470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934a94b-d9ca-4e30-8715-e5bcf448ffc1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:47.568306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd875f6-42ae-4601-9424-62639efcf849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:47.568977 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd875f6-42ae-4601-9424-62639efcf849 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:47.569262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd875f6-42ae-4601-9424-62639efcf849 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:47.569537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd875f6-42ae-4601-9424-62639efcf849 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:47.570218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd875f6-42ae-4601-9424-62639efcf849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:47.572183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd875f6-42ae-4601-9424-62639efcf849 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:47.647006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1307/2618] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:11:47 2022] POST /identity/v3/users => generated 301 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:47.648178 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa658999-1448-437c-91a6-4ad4cd5227ce admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:47.659698 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa658999-1448-437c-91a6-4ad4cd5227ce admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:47.664452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd5e13-b7b2-4e44-aab6-bc14f3d6a1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:47.665126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd5e13-b7b2-4e44-aab6-bc14f3d6a1af admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:47.665417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd5e13-b7b2-4e44-aab6-bc14f3d6a1af admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:47.665692 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd5e13-b7b2-4e44-aab6-bc14f3d6a1af admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:47.666429 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd5e13-b7b2-4e44-aab6-bc14f3d6a1af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:47.668388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd5e13-b7b2-4e44-aab6-bc14f3d6a1af admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:47.680087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1312/2619] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:11:47 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:47.684829 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd875f6-42ae-4601-9424-62639efcf849 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:47.695031 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd875f6-42ae-4601-9424-62639efcf849 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:47.701346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22770401-26a9-40ef-a7b5-6d5c51333429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:47.702551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22770401-26a9-40ef-a7b5-6d5c51333429 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:47.702883 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22770401-26a9-40ef-a7b5-6d5c51333429 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:47.703270 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22770401-26a9-40ef-a7b5-6d5c51333429 admin admin] PATH_INFO: `/v3/projects/60566de65d43480f90af797280211398/users/bbc28f045f534f7ab0651d4e1d5a201c/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:47.720760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22770401-26a9-40ef-a7b5-6d5c51333429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60566de65d43480f90af797280211398, user_id=bbc28f045f534f7ab0651d4e1d5a201c, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:47.723464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22770401-26a9-40ef-a7b5-6d5c51333429 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:47.849029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1308/2620] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:11:47 2022] PUT /identity/v3/projects/60566de65d43480f90af797280211398/users/bbc28f045f534f7ab0651d4e1d5a201c/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:47.852879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd5e13-b7b2-4e44-aab6-bc14f3d6a1af admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:47.863094 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd5e13-b7b2-4e44-aab6-bc14f3d6a1af admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:47.919431 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82def603-a22e-4ce7-9c0f-c87e035627bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:47.920101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82def603-a22e-4ce7-9c0f-c87e035627bf admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:47.920385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82def603-a22e-4ce7-9c0f-c87e035627bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:47.920663 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82def603-a22e-4ce7-9c0f-c87e035627bf admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:47.921388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82def603-a22e-4ce7-9c0f-c87e035627bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:47.923020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82def603-a22e-4ce7-9c0f-c87e035627bf admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:47.933218 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1313/2621] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:11:47 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:47.937380 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22770401-26a9-40ef-a7b5-6d5c51333429 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:47.948105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22770401-26a9-40ef-a7b5-6d5c51333429 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:47.952652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05dff1-5180-41ab-97ed-dd75ee2ee5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:47.953522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05dff1-5180-41ab-97ed-dd75ee2ee5ac admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:47.953798 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05dff1-5180-41ab-97ed-dd75ee2ee5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:47.954075 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05dff1-5180-41ab-97ed-dd75ee2ee5ac admin admin] PATH_INFO: `/v3/projects/60566de65d43480f90af797280211398/users/bbc28f045f534f7ab0651d4e1d5a201c/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:47.956013 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05dff1-5180-41ab-97ed-dd75ee2ee5ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=60566de65d43480f90af797280211398, user_id=bbc28f045f534f7ab0651d4e1d5a201c, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:47.958635 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05dff1-5180-41ab-97ed-dd75ee2ee5ac admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:47.969921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1309/2622] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:11:47 2022] PUT /identity/v3/projects/60566de65d43480f90af797280211398/users/bbc28f045f534f7ab0651d4e1d5a201c/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:47.980446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927b4f3-fe73-4172-8c1d-6b448222b776 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:47.980732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927b4f3-fe73-4172-8c1d-6b448222b776 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:47.981007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927b4f3-fe73-4172-8c1d-6b448222b776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:48.006753 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a927b4f3-fe73-4172-8c1d-6b448222b776 None None] MFA Rules not processed for user `bbc28f045f534f7ab0651d4e1d5a201c`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:48.007979 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a927b4f3-fe73-4172-8c1d-6b448222b776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:48.068717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:48.078781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1314/2623] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:11:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:48.089860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05dff1-5180-41ab-97ed-dd75ee2ee5ac admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:48.091584 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-de05dff1-5180-41ab-97ed-dd75ee2ee5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:48.170775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05dff1-5180-41ab-97ed-dd75ee2ee5ac admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:48.174455 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96625a2b-d869-4b77-a0f5-e2738af661b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:48.175137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96625a2b-d869-4b77-a0f5-e2738af661b2 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:48.175415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96625a2b-d869-4b77-a0f5-e2738af661b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:48.175691 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96625a2b-d869-4b77-a0f5-e2738af661b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:48.185727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96625a2b-d869-4b77-a0f5-e2738af661b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:48.187222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96625a2b-d869-4b77-a0f5-e2738af661b2 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:48.196429 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1310/2624] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:11:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:49.264887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927b4f3-fe73-4172-8c1d-6b448222b776 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:49.266302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a927b4f3-fe73-4172-8c1d-6b448222b776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:49.278963 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927b4f3-fe73-4172-8c1d-6b448222b776 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:49.282490 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45990764-ead8-4257-a50f-84d729eca554 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:49.283164 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45990764-ead8-4257-a50f-84d729eca554 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:49.283443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45990764-ead8-4257-a50f-84d729eca554 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:49.283714 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45990764-ead8-4257-a50f-84d729eca554 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:49.339711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45990764-ead8-4257-a50f-84d729eca554 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:49.341131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45990764-ead8-4257-a50f-84d729eca554 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:49.350146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1315/2625] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:11:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:54.301781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96625a2b-d869-4b77-a0f5-e2738af661b2 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:54.311417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96625a2b-d869-4b77-a0f5-e2738af661b2 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:54.315116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4e44f-e3e8-4b9b-a3cd-f321f543bb99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:54.315771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4e44f-e3e8-4b9b-a3cd-f321f543bb99 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:54.316048 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4e44f-e3e8-4b9b-a3cd-f321f543bb99 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:54.316319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4e44f-e3e8-4b9b-a3cd-f321f543bb99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:54.348258 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:54.375493 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4e44f-e3e8-4b9b-a3cd-f321f543bb99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:54.376940 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4e44f-e3e8-4b9b-a3cd-f321f543bb99 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:54.386550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1311/2626] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:11:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:58.132063 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45990764-ead8-4257-a50f-84d729eca554 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:58.196546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45990764-ead8-4257-a50f-84d729eca554 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:58.201269 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:58.202174 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:58.202453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:58.202727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] PATH_INFO: `/v3/users/3e14ea83f3ac4e5ca123382bfb8b5c9f` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:58.203955 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e14ea83f3ac4e5ca123382bfb8b5c9f)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:58.205900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:58.392736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e14ea83f3ac4e5ca123382bfb8b5c9f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:58.399515 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e14ea83f3ac4e5ca123382bfb8b5c9f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:58.400211 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e14ea83f3ac4e5ca123382bfb8b5c9f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:58.412454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e14ea83f3ac4e5ca123382bfb8b5c9f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:58.465288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e14ea83f3ac4e5ca123382bfb8b5c9f'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:58.471535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1316/2627] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:11:58 2022] DELETE /identity/v3/users/3e14ea83f3ac4e5ca123382bfb8b5c9f => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:58.475984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f4356-0515-4567-a28a-11696e062eb9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:58.482427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f4356-0515-4567-a28a-11696e062eb9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:58.482427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f4356-0515-4567-a28a-11696e062eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:58.494725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c05f4356-0515-4567-a28a-11696e062eb9 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:58.496093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c05f4356-0515-4567-a28a-11696e062eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:58.559337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1312/2628] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:58.569092 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:58.570633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:58.585148 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:58.627715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589a3f6-5680-481f-804f-60cdca817022 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:58.635533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be5eef-b807-45d1-9562-f1f974fc3122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:58.635533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be5eef-b807-45d1-9562-f1f974fc3122 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:58.635533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be5eef-b807-45d1-9562-f1f974fc3122 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:58.635533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be5eef-b807-45d1-9562-f1f974fc3122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:58.641804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be5eef-b807-45d1-9562-f1f974fc3122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:58.643280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be5eef-b807-45d1-9562-f1f974fc3122 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:58.653113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1317/2629] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:11:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:59.095985 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f4356-0515-4567-a28a-11696e062eb9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:59.099869 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c05f4356-0515-4567-a28a-11696e062eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:59.115023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f4356-0515-4567-a28a-11696e062eb9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:59.120556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:59.128660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:59.128660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:59.128660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] PATH_INFO: `/v3/projects/7e5ee1f4885e4a54af253e7986a17540` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:59.128660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e5ee1f4885e4a54af253e7986a17540)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:59.128660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:59.231320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e5ee1f4885e4a54af253e7986a17540', 'request_id': 'req-66b176f1-4058-4cc5-8c97-25448594d1d9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:59.287871 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e5ee1f4885e4a54af253e7986a17540', 'request_id': 'req-66b176f1-4058-4cc5-8c97-25448594d1d9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:59.294775 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e5ee1f4885e4a54af253e7986a17540', 'request_id': 'req-66b176f1-4058-4cc5-8c97-25448594d1d9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:59.305625 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] The token cache is being invalidate because project 7e5ee1f4885e4a54af253e7986a17540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:11:59.305909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:59.307061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:59.308598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1313/2630] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:11:59 2022] DELETE /identity/v3/projects/7e5ee1f4885e4a54af253e7986a17540 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:59.312546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be5eef-b807-45d1-9562-f1f974fc3122 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:59.324653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be5eef-b807-45d1-9562-f1f974fc3122 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:59.378265 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:59.379166 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:59.379444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:59.379712 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] PATH_INFO: `/v3/users/60c39215fb1a49bc85c652f78e36e279` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:59.380872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60c39215fb1a49bc85c652f78e36e279)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:59.382785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:59.518382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60c39215fb1a49bc85c652f78e36e279'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:59.525135 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60c39215fb1a49bc85c652f78e36e279'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:59.525839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60c39215fb1a49bc85c652f78e36e279'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:59.537383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60c39215fb1a49bc85c652f78e36e279'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:59.578999 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60c39215fb1a49bc85c652f78e36e279'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:59.585202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1318/2631] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:11:59 2022] DELETE /identity/v3/users/60c39215fb1a49bc85c652f78e36e279 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:59.981660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:59.983210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:59.997245 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b176f1-4058-4cc5-8c97-25448594d1d9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:00.048918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:00.055943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:00.056807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:00.057089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:00.057359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] PATH_INFO: `/v3/projects/4795844146324bb6a0c42c048481b52e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:00.058527 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4795844146324bb6a0c42c048481b52e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:00.060470 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:00.173710 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4795844146324bb6a0c42c048481b52e', 'request_id': 'req-0cc69bf4-8b87-4b68-a173-982805f958f3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:00.265090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4795844146324bb6a0c42c048481b52e', 'request_id': 'req-0cc69bf4-8b87-4b68-a173-982805f958f3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:00.271894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4795844146324bb6a0c42c048481b52e', 'request_id': 'req-0cc69bf4-8b87-4b68-a173-982805f958f3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:00.282874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] The token cache is being invalidate because project 4795844146324bb6a0c42c048481b52e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:00.283175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:00.283783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:00.285300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1314/2632] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:11:59 2022] DELETE /identity/v3/projects/4795844146324bb6a0c42c048481b52e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:00.289337 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:00.290719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:00.303678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecea872-6d2b-464f-b6a3-90483c790215 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:00.335598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:00.360203 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83713942-bd93-4615-8e33-63ce1d3460ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:00.368934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83713942-bd93-4615-8e33-63ce1d3460ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:00.368934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83713942-bd93-4615-8e33-63ce1d3460ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:00.368934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83713942-bd93-4615-8e33-63ce1d3460ad admin admin] PATH_INFO: `/v3/users/bbc28f045f534f7ab0651d4e1d5a201c` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:00.368934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83713942-bd93-4615-8e33-63ce1d3460ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbc28f045f534f7ab0651d4e1d5a201c)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:00.368934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83713942-bd93-4615-8e33-63ce1d3460ad admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:00.604096 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-83713942-bd93-4615-8e33-63ce1d3460ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbc28f045f534f7ab0651d4e1d5a201c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:00.610748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-83713942-bd93-4615-8e33-63ce1d3460ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbc28f045f534f7ab0651d4e1d5a201c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:00.611452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-83713942-bd93-4615-8e33-63ce1d3460ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbc28f045f534f7ab0651d4e1d5a201c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:00.623043 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-83713942-bd93-4615-8e33-63ce1d3460ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbc28f045f534f7ab0651d4e1d5a201c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:00.688110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-83713942-bd93-4615-8e33-63ce1d3460ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbc28f045f534f7ab0651d4e1d5a201c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:00.694504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1319/2633] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:12:00 2022] DELETE /identity/v3/users/bbc28f045f534f7ab0651d4e1d5a201c => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:01.141488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:01.143022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:01.162060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc69bf4-8b87-4b68-a173-982805f958f3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:01.233176 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90acd069-9295-45ec-b0e4-38df0ff24a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:01.235643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acd069-9295-45ec-b0e4-38df0ff24a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.235946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acd069-9295-45ec-b0e4-38df0ff24a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.236226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acd069-9295-45ec-b0e4-38df0ff24a13 admin admin] PATH_INFO: `/v3/projects/60566de65d43480f90af797280211398` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.239272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90acd069-9295-45ec-b0e4-38df0ff24a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60566de65d43480f90af797280211398)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:01.241233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90acd069-9295-45ec-b0e4-38df0ff24a13 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:01.368783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90acd069-9295-45ec-b0e4-38df0ff24a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60566de65d43480f90af797280211398', 'request_id': 'req-90acd069-9295-45ec-b0e4-38df0ff24a13'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.448164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90acd069-9295-45ec-b0e4-38df0ff24a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60566de65d43480f90af797280211398', 'request_id': 'req-90acd069-9295-45ec-b0e4-38df0ff24a13'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.455133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90acd069-9295-45ec-b0e4-38df0ff24a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60566de65d43480f90af797280211398', 'request_id': 'req-90acd069-9295-45ec-b0e4-38df0ff24a13'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.467289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90acd069-9295-45ec-b0e4-38df0ff24a13 admin admin] The token cache is being invalidate because project 60566de65d43480f90af797280211398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:01.467577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90acd069-9295-45ec-b0e4-38df0ff24a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.468241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90acd069-9295-45ec-b0e4-38df0ff24a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.469844 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1315/2634] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:12:01 2022] DELETE /identity/v3/projects/60566de65d43480f90af797280211398 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:01.476681 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc695eb-b6e0-45de-b2ba-0f2c4a2f1d6c None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.477200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc695eb-b6e0-45de-b2ba-0f2c4a2f1d6c None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.479067 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc695eb-b6e0-45de-b2ba-0f2c4a2f1d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.497501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5cc695eb-b6e0-45de-b2ba-0f2c4a2f1d6c None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:01.498724 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5cc695eb-b6e0-45de-b2ba-0f2c4a2f1d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:01.564151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1320/2635] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:01.576705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12fc59-5e45-4d33-9041-bbcef8a566de None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.576999 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12fc59-5e45-4d33-9041-bbcef8a566de None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.577280 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12fc59-5e45-4d33-9041-bbcef8a566de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.596799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2b12fc59-5e45-4d33-9041-bbcef8a566de None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:01.599517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2b12fc59-5e45-4d33-9041-bbcef8a566de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:01.606945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1316/2636] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:01.611261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc695eb-b6e0-45de-b2ba-0f2c4a2f1d6c None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:01.621493 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc695eb-b6e0-45de-b2ba-0f2c4a2f1d6c None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:01.626210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9b504-013a-4290-a5d1-e8123abda6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:01.626964 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9b504-013a-4290-a5d1-e8123abda6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.627246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9b504-013a-4290-a5d1-e8123abda6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.633889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9b504-013a-4290-a5d1-e8123abda6a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.633889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9b504-013a-4290-a5d1-e8123abda6a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:01.633889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9b504-013a-4290-a5d1-e8123abda6a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:01.633889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9b504-013a-4290-a5d1-e8123abda6a3 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:01.642525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1321/2637] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:12:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:12:01.648038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12fc59-5e45-4d33-9041-bbcef8a566de None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:01.658056 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12fc59-5e45-4d33-9041-bbcef8a566de None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:01.663727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2ebf-ab70-4be7-b70b-654812dcadc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:01.664425 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2ebf-ab70-4be7-b70b-654812dcadc4 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.664769 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2ebf-ab70-4be7-b70b-654812dcadc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.665218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2ebf-ab70-4be7-b70b-654812dcadc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.666085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c2ebf-ab70-4be7-b70b-654812dcadc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:01.668607 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c2ebf-ab70-4be7-b70b-654812dcadc4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:01.673377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:01.821490 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1317/2638] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:12:01 2022] POST /identity/v3/projects => generated 402 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:01.825664 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9b504-013a-4290-a5d1-e8123abda6a3 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:01.835958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9b504-013a-4290-a5d1-e8123abda6a3 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:01.877033 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:01.893360 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae447e9-69f6-4a5f-b3f4-015052e1fdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:01.894031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae447e9-69f6-4a5f-b3f4-015052e1fdce admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.894309 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae447e9-69f6-4a5f-b3f4-015052e1fdce admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.894587 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae447e9-69f6-4a5f-b3f4-015052e1fdce admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.895317 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae447e9-69f6-4a5f-b3f4-015052e1fdce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:01.897197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae447e9-69f6-4a5f-b3f4-015052e1fdce admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:01.974271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1322/2639] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:12:01 2022] POST /identity/v3/users => generated 302 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:12:01.978164 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2ebf-ab70-4be7-b70b-654812dcadc4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:01.989255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2ebf-ab70-4be7-b70b-654812dcadc4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:01.994322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc921d-47e8-482b-b698-8fc566700768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:01.995208 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc921d-47e8-482b-b698-8fc566700768 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.995782 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc921d-47e8-482b-b698-8fc566700768 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.996068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc921d-47e8-482b-b698-8fc566700768 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.996808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc921d-47e8-482b-b698-8fc566700768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:01.998399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc921d-47e8-482b-b698-8fc566700768 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.008901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1318/2640] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:01 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:02.013532 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae447e9-69f6-4a5f-b3f4-015052e1fdce admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.023670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae447e9-69f6-4a5f-b3f4-015052e1fdce admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.028990 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5caaa-8128-4266-ac9b-35db82808483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.029654 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5caaa-8128-4266-ac9b-35db82808483 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.029931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5caaa-8128-4266-ac9b-35db82808483 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.030200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5caaa-8128-4266-ac9b-35db82808483 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.030951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5caaa-8128-4266-ac9b-35db82808483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.032524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5caaa-8128-4266-ac9b-35db82808483 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.042700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1323/2641] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:02 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:02.046964 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc921d-47e8-482b-b698-8fc566700768 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.057506 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc921d-47e8-482b-b698-8fc566700768 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.062310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04f72a-b194-454a-ad8f-74df0681d081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.063226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04f72a-b194-454a-ad8f-74df0681d081 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.063508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04f72a-b194-454a-ad8f-74df0681d081 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.063854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04f72a-b194-454a-ad8f-74df0681d081 admin admin] PATH_INFO: `/v3/projects/385a0243ed1840388125138d6acca28a/users/b857d93053c94bffa545dd9ab78a19e0/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.079637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04f72a-b194-454a-ad8f-74df0681d081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=385a0243ed1840388125138d6acca28a, user_id=b857d93053c94bffa545dd9ab78a19e0, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.082204 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04f72a-b194-454a-ad8f-74df0681d081 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.162678 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1319/2642] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:12:02 2022] PUT /identity/v3/projects/385a0243ed1840388125138d6acca28a/users/b857d93053c94bffa545dd9ab78a19e0/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:02.174182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90a085-8991-41f8-b6c5-c723395b0c99 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.174471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90a085-8991-41f8-b6c5-c723395b0c99 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.174756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90a085-8991-41f8-b6c5-c723395b0c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.202985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5a90a085-8991-41f8-b6c5-c723395b0c99 None None] MFA Rules not processed for user `b857d93053c94bffa545dd9ab78a19e0`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:02.204227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5a90a085-8991-41f8-b6c5-c723395b0c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:02.267025 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1324/2643] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:02 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:02.271548 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04f72a-b194-454a-ad8f-74df0681d081 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.332221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04f72a-b194-454a-ad8f-74df0681d081 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.336782 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8f861-580a-4c82-bb9f-120b2c9c5745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.337433 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8f861-580a-4c82-bb9f-120b2c9c5745 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.337728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8f861-580a-4c82-bb9f-120b2c9c5745 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.338001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8f861-580a-4c82-bb9f-120b2c9c5745 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.338687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8f861-580a-4c82-bb9f-120b2c9c5745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.340598 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8f861-580a-4c82-bb9f-120b2c9c5745 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.438592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1320/2644] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:12:02 2022] POST /identity/v3/projects => generated 402 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:02.442461 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90a085-8991-41f8-b6c5-c723395b0c99 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.454368 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90a085-8991-41f8-b6c5-c723395b0c99 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.510086 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137233b-95ce-4fb4-abbb-8871af5a9438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.510928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137233b-95ce-4fb4-abbb-8871af5a9438 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.511063 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137233b-95ce-4fb4-abbb-8871af5a9438 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.511310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137233b-95ce-4fb4-abbb-8871af5a9438 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.511991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137233b-95ce-4fb4-abbb-8871af5a9438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.513880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137233b-95ce-4fb4-abbb-8871af5a9438 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.609009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1325/2645] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:12:02 2022] POST /identity/v3/users => generated 308 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:02.613024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8f861-580a-4c82-bb9f-120b2c9c5745 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.623913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8f861-580a-4c82-bb9f-120b2c9c5745 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.628629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa566f5-43f0-43cc-9ab9-fdb49946b42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.629290 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa566f5-43f0-43cc-9ab9-fdb49946b42a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.629591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa566f5-43f0-43cc-9ab9-fdb49946b42a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.629895 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa566f5-43f0-43cc-9ab9-fdb49946b42a admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.630768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa566f5-43f0-43cc-9ab9-fdb49946b42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.632395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa566f5-43f0-43cc-9ab9-fdb49946b42a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.643718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1321/2646] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:02 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:02.647962 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137233b-95ce-4fb4-abbb-8871af5a9438 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.658039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137233b-95ce-4fb4-abbb-8871af5a9438 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.662537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f05269-5939-496e-9c96-6fc7ffb55c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.663650 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f05269-5939-496e-9c96-6fc7ffb55c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.663929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f05269-5939-496e-9c96-6fc7ffb55c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.664200 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f05269-5939-496e-9c96-6fc7ffb55c40 admin admin] PATH_INFO: `/v3/projects/20111e3aca8148c5a602a9a7dbc6a425/users/b12b7fb3285d4780bed96b2e7a4760d8/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.679890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f05269-5939-496e-9c96-6fc7ffb55c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20111e3aca8148c5a602a9a7dbc6a425, user_id=b12b7fb3285d4780bed96b2e7a4760d8, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.683623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f05269-5939-496e-9c96-6fc7ffb55c40 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.739341 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1326/2647] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:12:02 2022] PUT /identity/v3/projects/20111e3aca8148c5a602a9a7dbc6a425/users/b12b7fb3285d4780bed96b2e7a4760d8/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:02.743383 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa566f5-43f0-43cc-9ab9-fdb49946b42a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.753921 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa566f5-43f0-43cc-9ab9-fdb49946b42a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.809560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff33998-97e6-47f9-a6ef-910dd46a4459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.810250 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff33998-97e6-47f9-a6ef-910dd46a4459 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.810531 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff33998-97e6-47f9-a6ef-910dd46a4459 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.810807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff33998-97e6-47f9-a6ef-910dd46a4459 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.811558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff33998-97e6-47f9-a6ef-910dd46a4459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.813174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff33998-97e6-47f9-a6ef-910dd46a4459 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.822874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1322/2648] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:02 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:02.827210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f05269-5939-496e-9c96-6fc7ffb55c40 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.837363 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f05269-5939-496e-9c96-6fc7ffb55c40 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.842011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10093d-2a79-4ad9-b292-8d4f6b2e67a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.842915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10093d-2a79-4ad9-b292-8d4f6b2e67a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.843195 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10093d-2a79-4ad9-b292-8d4f6b2e67a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.843466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10093d-2a79-4ad9-b292-8d4f6b2e67a1 admin admin] PATH_INFO: `/v3/projects/20111e3aca8148c5a602a9a7dbc6a425/users/b12b7fb3285d4780bed96b2e7a4760d8/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.845391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10093d-2a79-4ad9-b292-8d4f6b2e67a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20111e3aca8148c5a602a9a7dbc6a425, user_id=b12b7fb3285d4780bed96b2e7a4760d8, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.848068 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10093d-2a79-4ad9-b292-8d4f6b2e67a1 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.858776 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1327/2649] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:12:02 2022] PUT /identity/v3/projects/20111e3aca8148c5a602a9a7dbc6a425/users/b12b7fb3285d4780bed96b2e7a4760d8/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:02.869722 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679a21f-d262-4a65-8d85-4cc8d40a23e9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.870562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679a21f-d262-4a65-8d85-4cc8d40a23e9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.870875 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679a21f-d262-4a65-8d85-4cc8d40a23e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.896795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-0679a21f-d262-4a65-8d85-4cc8d40a23e9 None None] MFA Rules not processed for user `b12b7fb3285d4780bed96b2e7a4760d8`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:02.897990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0679a21f-d262-4a65-8d85-4cc8d40a23e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:02.907558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:02.967552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1323/2650] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:02 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:02.978719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10093d-2a79-4ad9-b292-8d4f6b2e67a1 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.980494 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-da10093d-2a79-4ad9-b292-8d4f6b2e67a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:02.985949 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:03.039241 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10093d-2a79-4ad9-b292-8d4f6b2e67a1 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:03.043466 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca3399-0ddf-4cd3-ac2c-302f10cb98c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:03.044113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca3399-0ddf-4cd3-ac2c-302f10cb98c4 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:03.044390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca3399-0ddf-4cd3-ac2c-302f10cb98c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:03.044660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca3399-0ddf-4cd3-ac2c-302f10cb98c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:03.054666 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca3399-0ddf-4cd3-ac2c-302f10cb98c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:03.056124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca3399-0ddf-4cd3-ac2c-302f10cb98c4 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:03.065551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1328/2651] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:12:02 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:03.339560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679a21f-d262-4a65-8d85-4cc8d40a23e9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:03.349292 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679a21f-d262-4a65-8d85-4cc8d40a23e9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:03.353061 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec2b31-4c93-47f0-aee1-7586728dcc4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:03.354557 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec2b31-4c93-47f0-aee1-7586728dcc4b service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:03.354868 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec2b31-4c93-47f0-aee1-7586728dcc4b service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:03.355146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec2b31-4c93-47f0-aee1-7586728dcc4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:03.411977 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec2b31-4c93-47f0-aee1-7586728dcc4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:03.413416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec2b31-4c93-47f0-aee1-7586728dcc4b service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:03.422767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1324/2652] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:12:03 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:08.728751 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca3399-0ddf-4cd3-ac2c-302f10cb98c4 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:08.793806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca3399-0ddf-4cd3-ac2c-302f10cb98c4 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:08.798285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:08.799189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:08.799468 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:08.799739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] PATH_INFO: `/v3/users/b857d93053c94bffa545dd9ab78a19e0` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:08.800906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b857d93053c94bffa545dd9ab78a19e0)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:08.802846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:09.023100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b857d93053c94bffa545dd9ab78a19e0'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:09.030400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b857d93053c94bffa545dd9ab78a19e0'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:09.031977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b857d93053c94bffa545dd9ab78a19e0'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:09.043436 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b857d93053c94bffa545dd9ab78a19e0'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:09.133895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b857d93053c94bffa545dd9ab78a19e0'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:09.141347 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1329/2653] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:12:08 2022] DELETE /identity/v3/users/b857d93053c94bffa545dd9ab78a19e0 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:09.145986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac7bd4-4c44-4046-b452-ce0f2f0820c8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:09.146455 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac7bd4-4c44-4046-b452-ce0f2f0820c8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:09.146570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac7bd4-4c44-4046-b452-ce0f2f0820c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:09.163797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-cdac7bd4-4c44-4046-b452-ce0f2f0820c8 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:09.164991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cdac7bd4-4c44-4046-b452-ce0f2f0820c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:09.225577 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1325/2654] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:09.235818 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:09.237231 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:09.267822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:09.298192 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d4c83-b2ef-41d6-a71a-567adb05041c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:09.302407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade2c54-7242-4ca4-b513-3993deb29482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:09.303113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade2c54-7242-4ca4-b513-3993deb29482 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:09.303403 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade2c54-7242-4ca4-b513-3993deb29482 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:09.303673 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade2c54-7242-4ca4-b513-3993deb29482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:09.314298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade2c54-7242-4ca4-b513-3993deb29482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:09.315775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade2c54-7242-4ca4-b513-3993deb29482 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:09.326260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1330/2655] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:09.730708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac7bd4-4c44-4046-b452-ce0f2f0820c8 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:09.732219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cdac7bd4-4c44-4046-b452-ce0f2f0820c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:09.744486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac7bd4-4c44-4046-b452-ce0f2f0820c8 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:09.755390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:09.756480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:09.756772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:09.757051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] PATH_INFO: `/v3/projects/385a0243ed1840388125138d6acca28a` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:09.758277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=385a0243ed1840388125138d6acca28a)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:09.760294 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:09.895912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:09.902031 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '385a0243ed1840388125138d6acca28a', 'request_id': 'req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:09.974266 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '385a0243ed1840388125138d6acca28a', 'request_id': 'req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:09.981154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '385a0243ed1840388125138d6acca28a', 'request_id': 'req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:09.993944 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] The token cache is being invalidate because project 385a0243ed1840388125138d6acca28a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:09.994476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:09.996160 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:09.997816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1326/2656] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:12:09 2022] DELETE /identity/v3/projects/385a0243ed1840388125138d6acca28a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:10.002119 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade2c54-7242-4ca4-b513-3993deb29482 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:10.012929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade2c54-7242-4ca4-b513-3993deb29482 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:10.074362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62b632-0c63-4657-8fc2-f65cf428ccd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:10.075274 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62b632-0c63-4657-8fc2-f65cf428ccd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:10.075553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62b632-0c63-4657-8fc2-f65cf428ccd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:10.075828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62b632-0c63-4657-8fc2-f65cf428ccd8 admin admin] PATH_INFO: `/v3/users/b12b7fb3285d4780bed96b2e7a4760d8` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:10.077060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62b632-0c63-4657-8fc2-f65cf428ccd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12b7fb3285d4780bed96b2e7a4760d8)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:10.079104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62b632-0c63-4657-8fc2-f65cf428ccd8 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:10.257842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0b62b632-0c63-4657-8fc2-f65cf428ccd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12b7fb3285d4780bed96b2e7a4760d8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:10.267020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0b62b632-0c63-4657-8fc2-f65cf428ccd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12b7fb3285d4780bed96b2e7a4760d8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:10.268074 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0b62b632-0c63-4657-8fc2-f65cf428ccd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12b7fb3285d4780bed96b2e7a4760d8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:10.283302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0b62b632-0c63-4657-8fc2-f65cf428ccd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12b7fb3285d4780bed96b2e7a4760d8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:10.344972 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0b62b632-0c63-4657-8fc2-f65cf428ccd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12b7fb3285d4780bed96b2e7a4760d8'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:10.354613 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1331/2657] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:12:10 2022] DELETE /identity/v3/users/b12b7fb3285d4780bed96b2e7a4760d8 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:10.821102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:10.822570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:10.835952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b2e5e-7bf0-4dfe-b404-0cd096034a38 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:10.896103 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:10.896978 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:10.897264 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:10.897542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb admin admin] PATH_INFO: `/v3/projects/20111e3aca8148c5a602a9a7dbc6a425` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:10.898728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=20111e3aca8148c5a602a9a7dbc6a425)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:10.900682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:10.992938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20111e3aca8148c5a602a9a7dbc6a425', 'request_id': 'req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.082901 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20111e3aca8148c5a602a9a7dbc6a425', 'request_id': 'req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.090842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20111e3aca8148c5a602a9a7dbc6a425', 'request_id': 'req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.102309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb admin admin] The token cache is being invalidate because project 20111e3aca8148c5a602a9a7dbc6a425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:11.102599 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.103367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4b53c1f8-c424-447d-ad7c-131a20f0b2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.104909 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1327/2658] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:12:10 2022] DELETE /identity/v3/projects/20111e3aca8148c5a602a9a7dbc6a425 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:11.110083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d0cb1-78bb-4c64-9c93-abbd7fcb1629 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.110538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d0cb1-78bb-4c64-9c93-abbd7fcb1629 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.110671 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d0cb1-78bb-4c64-9c93-abbd7fcb1629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.129252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-f53d0cb1-78bb-4c64-9c93-abbd7fcb1629 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:11.130464 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f53d0cb1-78bb-4c64-9c93-abbd7fcb1629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:11.175656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:11.193407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1332/2659] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:11.202077 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c949b9-a6d3-4924-90b8-cc5adae1a21b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.202424 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c949b9-a6d3-4924-90b8-cc5adae1a21b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.204871 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c949b9-a6d3-4924-90b8-cc5adae1a21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.222313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f7c949b9-a6d3-4924-90b8-cc5adae1a21b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:11.223595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f7c949b9-a6d3-4924-90b8-cc5adae1a21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:11.232570 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1328/2660] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:12:11.236977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d0cb1-78bb-4c64-9c93-abbd7fcb1629 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.250378 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d0cb1-78bb-4c64-9c93-abbd7fcb1629 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:11.257239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4496898-f1c1-476e-a440-bc4840ac30fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:11.258191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4496898-f1c1-476e-a440-bc4840ac30fe admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.258625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4496898-f1c1-476e-a440-bc4840ac30fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.259083 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4496898-f1c1-476e-a440-bc4840ac30fe admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.260123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4496898-f1c1-476e-a440-bc4840ac30fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:11.260623 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4496898-f1c1-476e-a440-bc4840ac30fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:11.263740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4496898-f1c1-476e-a440-bc4840ac30fe admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:11.282578 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1333/2661] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:12:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:12:11.287354 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c949b9-a6d3-4924-90b8-cc5adae1a21b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.301896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c949b9-a6d3-4924-90b8-cc5adae1a21b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:11.309167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7093c1-fc67-4d3c-ae34-999437a7aa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:11.310018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7093c1-fc67-4d3c-ae34-999437a7aa45 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.310303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7093c1-fc67-4d3c-ae34-999437a7aa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.310576 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7093c1-fc67-4d3c-ae34-999437a7aa45 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.311290 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7093c1-fc67-4d3c-ae34-999437a7aa45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:11.313193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7093c1-fc67-4d3c-ae34-999437a7aa45 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:11.398882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1329/2662] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:12:11 2022] POST /identity/v3/projects => generated 376 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:11.403678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4496898-f1c1-476e-a440-bc4840ac30fe admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.413745 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4496898-f1c1-476e-a440-bc4840ac30fe admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:11.471923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0898f68-3665-4cd9-8cde-e55b317a5702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:11.472604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0898f68-3665-4cd9-8cde-e55b317a5702 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.472890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0898f68-3665-4cd9-8cde-e55b317a5702 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.473165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0898f68-3665-4cd9-8cde-e55b317a5702 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.473898 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0898f68-3665-4cd9-8cde-e55b317a5702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:11.475935 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0898f68-3665-4cd9-8cde-e55b317a5702 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:11.597370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1334/2663] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:12:11 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:12:11.601694 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7093c1-fc67-4d3c-ae34-999437a7aa45 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.612741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7093c1-fc67-4d3c-ae34-999437a7aa45 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:11.617558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0648f1f-fa9f-4db8-ac0f-8793ba509bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:11.618282 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0648f1f-fa9f-4db8-ac0f-8793ba509bbf admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.618566 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0648f1f-fa9f-4db8-ac0f-8793ba509bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.618878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0648f1f-fa9f-4db8-ac0f-8793ba509bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.619615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0648f1f-fa9f-4db8-ac0f-8793ba509bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:11.621961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0648f1f-fa9f-4db8-ac0f-8793ba509bbf admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:11.633687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1330/2664] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:11 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:11.637704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0898f68-3665-4cd9-8cde-e55b317a5702 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.650036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0898f68-3665-4cd9-8cde-e55b317a5702 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:11.654753 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753fc4a-a059-4c3d-a330-12dd1733e675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:11.655451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753fc4a-a059-4c3d-a330-12dd1733e675 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.655756 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753fc4a-a059-4c3d-a330-12dd1733e675 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.656028 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753fc4a-a059-4c3d-a330-12dd1733e675 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.656750 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753fc4a-a059-4c3d-a330-12dd1733e675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:11.658425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753fc4a-a059-4c3d-a330-12dd1733e675 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:11.669774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1335/2665] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:11 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:11.673980 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0648f1f-fa9f-4db8-ac0f-8793ba509bbf admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.685203 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0648f1f-fa9f-4db8-ac0f-8793ba509bbf admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:11.689845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc070d2c-75d1-41b1-b31b-04bc1e03c026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:11.690730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc070d2c-75d1-41b1-b31b-04bc1e03c026 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.691051 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc070d2c-75d1-41b1-b31b-04bc1e03c026 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.691327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc070d2c-75d1-41b1-b31b-04bc1e03c026 admin admin] PATH_INFO: `/v3/projects/4f311538c5de441c99e04c80122dd113/users/d6bdda568a104e2db566737343d50b4c/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.706565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc070d2c-75d1-41b1-b31b-04bc1e03c026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f311538c5de441c99e04c80122dd113, user_id=d6bdda568a104e2db566737343d50b4c, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:11.709265 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc070d2c-75d1-41b1-b31b-04bc1e03c026 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:11.712419 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:11.814592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1331/2666] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:12:11 2022] PUT /identity/v3/projects/4f311538c5de441c99e04c80122dd113/users/d6bdda568a104e2db566737343d50b4c/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:11.825633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac87d8-b8d7-49cf-b6bf-611cd5f4a72f None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.825633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac87d8-b8d7-49cf-b6bf-611cd5f4a72f None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.826033 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac87d8-b8d7-49cf-b6bf-611cd5f4a72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.853541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-95ac87d8-b8d7-49cf-b6bf-611cd5f4a72f None None] MFA Rules not processed for user `d6bdda568a104e2db566737343d50b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:11.854753 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-95ac87d8-b8d7-49cf-b6bf-611cd5f4a72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:11.921899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1336/2667] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:11.926361 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc070d2c-75d1-41b1-b31b-04bc1e03c026 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.991090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc070d2c-75d1-41b1-b31b-04bc1e03c026 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:11.995842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b786f-026a-450d-b16c-335c52f68189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:11.996818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b786f-026a-450d-b16c-335c52f68189 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.997105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b786f-026a-450d-b16c-335c52f68189 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.997392 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b786f-026a-450d-b16c-335c52f68189 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.998086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b786f-026a-450d-b16c-335c52f68189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.000059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b786f-026a-450d-b16c-335c52f68189 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.118543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1332/2668] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:12:11 2022] POST /identity/v3/projects => generated 376 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:12.123008 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac87d8-b8d7-49cf-b6bf-611cd5f4a72f None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.135268 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac87d8-b8d7-49cf-b6bf-611cd5f4a72f None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.182727 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:12.199280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89b23b-c524-4547-8599-fbb1b3326f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.199976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b23b-c524-4547-8599-fbb1b3326f23 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.200256 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b23b-c524-4547-8599-fbb1b3326f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.200533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b23b-c524-4547-8599-fbb1b3326f23 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.201233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89b23b-c524-4547-8599-fbb1b3326f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.203248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89b23b-c524-4547-8599-fbb1b3326f23 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.328482 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1337/2669] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:12:12 2022] POST /identity/v3/users => generated 295 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:12.332816 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b786f-026a-450d-b16c-335c52f68189 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.344150 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b786f-026a-450d-b16c-335c52f68189 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.349094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab2a89-ab47-4cb8-9f54-10fe736d768e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.350231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab2a89-ab47-4cb8-9f54-10fe736d768e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.350231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab2a89-ab47-4cb8-9f54-10fe736d768e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.350231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab2a89-ab47-4cb8-9f54-10fe736d768e admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.354697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab2a89-ab47-4cb8-9f54-10fe736d768e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.354697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab2a89-ab47-4cb8-9f54-10fe736d768e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.365227 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1333/2670] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:12 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:12.369543 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89b23b-c524-4547-8599-fbb1b3326f23 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.379883 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89b23b-c524-4547-8599-fbb1b3326f23 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.384560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46e553-fd58-4087-b5a8-bfa695c9a0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.388847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46e553-fd58-4087-b5a8-bfa695c9a0cf admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.388847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46e553-fd58-4087-b5a8-bfa695c9a0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.388847 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46e553-fd58-4087-b5a8-bfa695c9a0cf admin admin] PATH_INFO: `/v3/projects/d2de04fae04f438685432638c8722836/users/308503c45fd042af9010324b5b7c3fd0/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.402776 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46e553-fd58-4087-b5a8-bfa695c9a0cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2de04fae04f438685432638c8722836, user_id=308503c45fd042af9010324b5b7c3fd0, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.405559 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46e553-fd58-4087-b5a8-bfa695c9a0cf admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.517522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1338/2671] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:12:12 2022] PUT /identity/v3/projects/d2de04fae04f438685432638c8722836/users/308503c45fd042af9010324b5b7c3fd0/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:12.521488 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab2a89-ab47-4cb8-9f54-10fe736d768e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.534205 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab2a89-ab47-4cb8-9f54-10fe736d768e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.592277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acaf8b7-ff1a-4f08-a2d5-74bfe18e2345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.592961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaf8b7-ff1a-4f08-a2d5-74bfe18e2345 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.593237 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaf8b7-ff1a-4f08-a2d5-74bfe18e2345 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.593512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaf8b7-ff1a-4f08-a2d5-74bfe18e2345 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.594235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaf8b7-ff1a-4f08-a2d5-74bfe18e2345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.595871 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaf8b7-ff1a-4f08-a2d5-74bfe18e2345 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.606271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1334/2672] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:12 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:12.610594 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46e553-fd58-4087-b5a8-bfa695c9a0cf admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.621460 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46e553-fd58-4087-b5a8-bfa695c9a0cf admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.626099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad17a7-1c4c-4ebf-88a7-5606ba6c6086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.627110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad17a7-1c4c-4ebf-88a7-5606ba6c6086 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.627311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad17a7-1c4c-4ebf-88a7-5606ba6c6086 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.627585 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad17a7-1c4c-4ebf-88a7-5606ba6c6086 admin admin] PATH_INFO: `/v3/projects/d2de04fae04f438685432638c8722836/users/308503c45fd042af9010324b5b7c3fd0/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.629537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad17a7-1c4c-4ebf-88a7-5606ba6c6086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2de04fae04f438685432638c8722836, user_id=308503c45fd042af9010324b5b7c3fd0, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.632215 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad17a7-1c4c-4ebf-88a7-5606ba6c6086 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.643553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1339/2673] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:12:12 2022] PUT /identity/v3/projects/d2de04fae04f438685432638c8722836/users/308503c45fd042af9010324b5b7c3fd0/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:12.655320 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be82499-bef9-446f-aa2e-8c7584f1d155 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.655633 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be82499-bef9-446f-aa2e-8c7584f1d155 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.655917 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be82499-bef9-446f-aa2e-8c7584f1d155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.686060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9be82499-bef9-446f-aa2e-8c7584f1d155 None None] MFA Rules not processed for user `308503c45fd042af9010324b5b7c3fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:12.687322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9be82499-bef9-446f-aa2e-8c7584f1d155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:12.739732 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:12.762449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1335/2674] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:12:12.773251 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad17a7-1c4c-4ebf-88a7-5606ba6c6086 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.775004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b9ad17a7-1c4c-4ebf-88a7-5606ba6c6086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:12.836300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad17a7-1c4c-4ebf-88a7-5606ba6c6086 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.839966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b18ba1-3aca-420c-b27b-6fbde799bf3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.840614 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b18ba1-3aca-420c-b27b-6fbde799bf3c service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.840895 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b18ba1-3aca-420c-b27b-6fbde799bf3c service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.841167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b18ba1-3aca-420c-b27b-6fbde799bf3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.899225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b18ba1-3aca-420c-b27b-6fbde799bf3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.900655 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b18ba1-3aca-420c-b27b-6fbde799bf3c service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.909718 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1340/2675] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:12:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:16.210316 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be82499-bef9-446f-aa2e-8c7584f1d155 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:16.272896 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be82499-bef9-446f-aa2e-8c7584f1d155 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:16.277289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:16.278168 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:16.278446 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:16.278718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] PATH_INFO: `/v3/users/d6bdda568a104e2db566737343d50b4c` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:16.280158 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6bdda568a104e2db566737343d50b4c)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:16.282075 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:16.444795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6bdda568a104e2db566737343d50b4c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.451820 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6bdda568a104e2db566737343d50b4c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.452530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6bdda568a104e2db566737343d50b4c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.464100 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6bdda568a104e2db566737343d50b4c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.545681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6bdda568a104e2db566737343d50b4c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.552188 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1336/2676] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:12:16 2022] DELETE /identity/v3/users/d6bdda568a104e2db566737343d50b4c => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:16.557220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b68e60-130a-4261-9553-3c787cb29f60 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:16.557913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b68e60-130a-4261-9553-3c787cb29f60 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:16.557913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b68e60-130a-4261-9553-3c787cb29f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:16.575065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c4b68e60-130a-4261-9553-3c787cb29f60 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:16.576323 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c4b68e60-130a-4261-9553-3c787cb29f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:16.620439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:16.638069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1341/2677] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:16.648188 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:16.649563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:16.708719 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181e615-522a-49e2-aafd-307aab5106a4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:16.712480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f559d27-5db8-4bb6-8410-7f54296f0fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:16.713127 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f559d27-5db8-4bb6-8410-7f54296f0fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:16.713408 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f559d27-5db8-4bb6-8410-7f54296f0fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:16.713683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f559d27-5db8-4bb6-8410-7f54296f0fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:16.723467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f559d27-5db8-4bb6-8410-7f54296f0fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:16.724976 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f559d27-5db8-4bb6-8410-7f54296f0fe5 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:16.735841 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1337/2678] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:17.118047 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b68e60-130a-4261-9553-3c787cb29f60 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:17.124729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c4b68e60-130a-4261-9553-3c787cb29f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:17.132526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b68e60-130a-4261-9553-3c787cb29f60 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:17.136616 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:17.137485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:17.137763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:17.138565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] PATH_INFO: `/v3/projects/4f311538c5de441c99e04c80122dd113` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:17.139815 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f311538c5de441c99e04c80122dd113)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:17.141713 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:17.248261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f311538c5de441c99e04c80122dd113', 'request_id': 'req-ab9acbcf-4a1e-4149-92aa-8307df06a23c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:17.333826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f311538c5de441c99e04c80122dd113', 'request_id': 'req-ab9acbcf-4a1e-4149-92aa-8307df06a23c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:17.343798 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f311538c5de441c99e04c80122dd113', 'request_id': 'req-ab9acbcf-4a1e-4149-92aa-8307df06a23c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:17.357112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] The token cache is being invalidate because project 4f311538c5de441c99e04c80122dd113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:17.357642 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:17.358359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:17.360057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1342/2679] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:12:17 2022] DELETE /identity/v3/projects/4f311538c5de441c99e04c80122dd113 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:17.364420 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f559d27-5db8-4bb6-8410-7f54296f0fe5 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:17.374881 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f559d27-5db8-4bb6-8410-7f54296f0fe5 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:17.383889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:17.433142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbd733-ee70-43d7-bc68-d72826b1e823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:17.434024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbd733-ee70-43d7-bc68-d72826b1e823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:17.434307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbd733-ee70-43d7-bc68-d72826b1e823 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:17.434584 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbd733-ee70-43d7-bc68-d72826b1e823 admin admin] PATH_INFO: `/v3/users/308503c45fd042af9010324b5b7c3fd0` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:17.436724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cbd733-ee70-43d7-bc68-d72826b1e823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=308503c45fd042af9010324b5b7c3fd0)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:17.439045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cbd733-ee70-43d7-bc68-d72826b1e823 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:17.723948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c0cbd733-ee70-43d7-bc68-d72826b1e823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '308503c45fd042af9010324b5b7c3fd0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:17.730938 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c0cbd733-ee70-43d7-bc68-d72826b1e823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '308503c45fd042af9010324b5b7c3fd0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:17.731594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c0cbd733-ee70-43d7-bc68-d72826b1e823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '308503c45fd042af9010324b5b7c3fd0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:17.744708 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c0cbd733-ee70-43d7-bc68-d72826b1e823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '308503c45fd042af9010324b5b7c3fd0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:17.853854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c0cbd733-ee70-43d7-bc68-d72826b1e823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '308503c45fd042af9010324b5b7c3fd0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:17.861643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1338/2680] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:12:17 2022] DELETE /identity/v3/users/308503c45fd042af9010324b5b7c3fd0 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:18.272445 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:18.273899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:18.289742 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9acbcf-4a1e-4149-92aa-8307df06a23c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:18.346992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:18.347859 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:18.348141 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:18.348414 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966 admin admin] PATH_INFO: `/v3/projects/d2de04fae04f438685432638c8722836` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:18.349579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2de04fae04f438685432638c8722836)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:18.351505 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:18.472619 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2de04fae04f438685432638c8722836', 'request_id': 'req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.539038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2de04fae04f438685432638c8722836', 'request_id': 'req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.545760 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2de04fae04f438685432638c8722836', 'request_id': 'req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.556448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966 admin admin] The token cache is being invalidate because project d2de04fae04f438685432638c8722836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:18.556733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.557414 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bd8d7af5-cbb8-4f30-95b2-1f01749b5966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.559003 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1343/2681] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:12:18 2022] DELETE /identity/v3/projects/d2de04fae04f438685432638c8722836 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:18.564524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248bd9e-89fb-449f-bc99-909e8fd7a089 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:18.565023 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248bd9e-89fb-449f-bc99-909e8fd7a089 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:18.565137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248bd9e-89fb-449f-bc99-909e8fd7a089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:18.584451 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5248bd9e-89fb-449f-bc99-909e8fd7a089 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:18.585634 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5248bd9e-89fb-449f-bc99-909e8fd7a089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:18.643472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1339/2682] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:18.651514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832e08f-1b0a-4266-83e7-69f7ac6a82ec None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:18.651804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832e08f-1b0a-4266-83e7-69f7ac6a82ec None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:18.652085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832e08f-1b0a-4266-83e7-69f7ac6a82ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:18.659162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:18.671444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-4832e08f-1b0a-4266-83e7-69f7ac6a82ec None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:18.672752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4832e08f-1b0a-4266-83e7-69f7ac6a82ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:18.681452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1344/2683] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:18.685381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248bd9e-89fb-449f-bc99-909e8fd7a089 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:18.695699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248bd9e-89fb-449f-bc99-909e8fd7a089 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:18.700293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715737fe-f0db-472b-abb1-c0acea4ed426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:18.700948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715737fe-f0db-472b-abb1-c0acea4ed426 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:18.701234 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715737fe-f0db-472b-abb1-c0acea4ed426 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:18.701508 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715737fe-f0db-472b-abb1-c0acea4ed426 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:18.702813 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715737fe-f0db-472b-abb1-c0acea4ed426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:18.703142 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715737fe-f0db-472b-abb1-c0acea4ed426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:18.704729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715737fe-f0db-472b-abb1-c0acea4ed426 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:18.716448 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1340/2684] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:12:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:12:18.721285 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832e08f-1b0a-4266-83e7-69f7ac6a82ec None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:18.730528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832e08f-1b0a-4266-83e7-69f7ac6a82ec None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:18.736770 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed1806-edb3-42ea-8d8c-1c653f9e2ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:18.737432 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed1806-edb3-42ea-8d8c-1c653f9e2ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:18.737715 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed1806-edb3-42ea-8d8c-1c653f9e2ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:18.737990 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed1806-edb3-42ea-8d8c-1c653f9e2ba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:18.738688 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed1806-edb3-42ea-8d8c-1c653f9e2ba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:18.740644 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed1806-edb3-42ea-8d8c-1c653f9e2ba4 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:18.821604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1345/2685] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:12:18 2022] POST /identity/v3/projects => generated 376 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:12:18.825699 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715737fe-f0db-472b-abb1-c0acea4ed426 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:18.835594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715737fe-f0db-472b-abb1-c0acea4ed426 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:18.854360 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:18.893424 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2991fe-1254-4ba7-93b4-92c7fa72ac82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:18.894090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2991fe-1254-4ba7-93b4-92c7fa72ac82 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:18.894367 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2991fe-1254-4ba7-93b4-92c7fa72ac82 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:18.894640 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2991fe-1254-4ba7-93b4-92c7fa72ac82 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:18.895353 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2991fe-1254-4ba7-93b4-92c7fa72ac82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:18.897251 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2991fe-1254-4ba7-93b4-92c7fa72ac82 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:19.020671 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1341/2686] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:12:18 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:19.024799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed1806-edb3-42ea-8d8c-1c653f9e2ba4 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:19.036148 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed1806-edb3-42ea-8d8c-1c653f9e2ba4 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:19.040951 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f2260-c470-4f86-b129-5b8aa8839335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:19.041615 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f2260-c470-4f86-b129-5b8aa8839335 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:19.041900 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f2260-c470-4f86-b129-5b8aa8839335 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:19.042175 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f2260-c470-4f86-b129-5b8aa8839335 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:19.042930 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f2260-c470-4f86-b129-5b8aa8839335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:19.044546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f2260-c470-4f86-b129-5b8aa8839335 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:19.055369 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1346/2687] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:19 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:19.059507 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2991fe-1254-4ba7-93b4-92c7fa72ac82 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:19.069373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2991fe-1254-4ba7-93b4-92c7fa72ac82 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:19.074010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee5d63-1aa1-4c02-a291-ad4589a9d484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:19.074608 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee5d63-1aa1-4c02-a291-ad4589a9d484 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:19.074933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee5d63-1aa1-4c02-a291-ad4589a9d484 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:19.075210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee5d63-1aa1-4c02-a291-ad4589a9d484 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:19.075927 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee5d63-1aa1-4c02-a291-ad4589a9d484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:19.078137 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee5d63-1aa1-4c02-a291-ad4589a9d484 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:19.088161 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1342/2688] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:19 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:19.092239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f2260-c470-4f86-b129-5b8aa8839335 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:19.103058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f2260-c470-4f86-b129-5b8aa8839335 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:19.107775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bb5f2-0330-47f0-b594-2ed2a193a78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:19.108885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bb5f2-0330-47f0-b594-2ed2a193a78c admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:19.109164 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bb5f2-0330-47f0-b594-2ed2a193a78c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:19.109443 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bb5f2-0330-47f0-b594-2ed2a193a78c admin admin] PATH_INFO: `/v3/projects/dfc12cd21b9543afb9b29ecb72ce5681/users/9100c3a456664d23950dcffe835a89da/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:19.124170 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bb5f2-0330-47f0-b594-2ed2a193a78c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc12cd21b9543afb9b29ecb72ce5681, user_id=9100c3a456664d23950dcffe835a89da, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:19.126853 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bb5f2-0330-47f0-b594-2ed2a193a78c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:19.220126 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1347/2689] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:12:19 2022] PUT /identity/v3/projects/dfc12cd21b9543afb9b29ecb72ce5681/users/9100c3a456664d23950dcffe835a89da/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:19.231321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fdd03-3949-4f86-b1e8-81521112bff9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:19.232828 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fdd03-3949-4f86-b1e8-81521112bff9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:19.232828 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fdd03-3949-4f86-b1e8-81521112bff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:19.259366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-835fdd03-3949-4f86-b1e8-81521112bff9 None None] MFA Rules not processed for user `9100c3a456664d23950dcffe835a89da`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:19.260582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-835fdd03-3949-4f86-b1e8-81521112bff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:19.325008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1343/2690] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:19.336817 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bb5f2-0330-47f0-b594-2ed2a193a78c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:19.338501 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-321bb5f2-0330-47f0-b594-2ed2a193a78c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:19.394624 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bb5f2-0330-47f0-b594-2ed2a193a78c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:19.398291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a6066-e21d-4d13-a25c-076ae85e2363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:19.398965 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a6066-e21d-4d13-a25c-076ae85e2363 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:19.399248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a6066-e21d-4d13-a25c-076ae85e2363 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:19.399526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a6066-e21d-4d13-a25c-076ae85e2363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:19.408571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a6066-e21d-4d13-a25c-076ae85e2363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:19.409993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a6066-e21d-4d13-a25c-076ae85e2363 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:19.423179 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1348/2691] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:12:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:29.322028 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:29.414830 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:45.533413 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fdd03-3949-4f86-b1e8-81521112bff9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:45.597385 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fdd03-3949-4f86-b1e8-81521112bff9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:45.602022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:45.602934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:45.603210 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:45.603481 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] PATH_INFO: `/v3/users/9100c3a456664d23950dcffe835a89da` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:45.604857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9100c3a456664d23950dcffe835a89da)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:45.606882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:45.797919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9100c3a456664d23950dcffe835a89da'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:45.804879 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9100c3a456664d23950dcffe835a89da'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:45.805532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9100c3a456664d23950dcffe835a89da'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:45.817861 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9100c3a456664d23950dcffe835a89da'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:45.876105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9100c3a456664d23950dcffe835a89da'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:45.882441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1344/2692] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:12:45 2022] DELETE /identity/v3/users/9100c3a456664d23950dcffe835a89da => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:45.887927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67965e07-ff2c-4fea-a230-e5b36ab6af94 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:45.888457 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67965e07-ff2c-4fea-a230-e5b36ab6af94 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:45.888565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67965e07-ff2c-4fea-a230-e5b36ab6af94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:45.905981 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-67965e07-ff2c-4fea-a230-e5b36ab6af94 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:45.907214 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-67965e07-ff2c-4fea-a230-e5b36ab6af94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:45.969974 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1349/2693] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:45.979614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:45.981001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:46.037899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de6d2a-ebd7-4afe-b039-24bebe93aee3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:46.041824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb2033-cde4-40ae-b01b-505d14c63863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:46.042551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb2033-cde4-40ae-b01b-505d14c63863 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:46.042858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb2033-cde4-40ae-b01b-505d14c63863 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:46.043140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb2033-cde4-40ae-b01b-505d14c63863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:46.052409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb2033-cde4-40ae-b01b-505d14c63863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:46.053758 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb2033-cde4-40ae-b01b-505d14c63863 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:46.064244 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1345/2694] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:46.445865 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67965e07-ff2c-4fea-a230-e5b36ab6af94 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:46.447311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-67965e07-ff2c-4fea-a230-e5b36ab6af94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:46.460507 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67965e07-ff2c-4fea-a230-e5b36ab6af94 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:46.465649 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d368-4704-4122-8912-60456b38b098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:46.466525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d368-4704-4122-8912-60456b38b098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:46.466852 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d368-4704-4122-8912-60456b38b098 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:46.467171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d368-4704-4122-8912-60456b38b098 admin admin] PATH_INFO: `/v3/projects/dfc12cd21b9543afb9b29ecb72ce5681` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:46.468356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251d368-4704-4122-8912-60456b38b098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfc12cd21b9543afb9b29ecb72ce5681)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:46.470441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251d368-4704-4122-8912-60456b38b098 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:46.580334 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6251d368-4704-4122-8912-60456b38b098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfc12cd21b9543afb9b29ecb72ce5681', 'request_id': 'req-6251d368-4704-4122-8912-60456b38b098'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:46.636799 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6251d368-4704-4122-8912-60456b38b098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfc12cd21b9543afb9b29ecb72ce5681', 'request_id': 'req-6251d368-4704-4122-8912-60456b38b098'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:46.644133 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6251d368-4704-4122-8912-60456b38b098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfc12cd21b9543afb9b29ecb72ce5681', 'request_id': 'req-6251d368-4704-4122-8912-60456b38b098'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:46.655826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6251d368-4704-4122-8912-60456b38b098 admin admin] The token cache is being invalidate because project dfc12cd21b9543afb9b29ecb72ce5681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:46.656371 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6251d368-4704-4122-8912-60456b38b098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:46.657027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-6251d368-4704-4122-8912-60456b38b098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:46.658721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1350/2695] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:12:46 2022] DELETE /identity/v3/projects/dfc12cd21b9543afb9b29ecb72ce5681 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:46.663712 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f2cd2-5fed-44ee-8d41-1dc111631960 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:46.664186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f2cd2-5fed-44ee-8d41-1dc111631960 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:46.664388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f2cd2-5fed-44ee-8d41-1dc111631960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:46.683925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bd8f2cd2-5fed-44ee-8d41-1dc111631960 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:46.684951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bd8f2cd2-5fed-44ee-8d41-1dc111631960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:46.702319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:46.747263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1346/2696] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:46.755213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832745a-df6e-47af-bf0f-eef3ff2e2648 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:46.755532 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832745a-df6e-47af-bf0f-eef3ff2e2648 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:46.755775 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832745a-df6e-47af-bf0f-eef3ff2e2648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:46.774367 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-e832745a-df6e-47af-bf0f-eef3ff2e2648 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:46.775591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e832745a-df6e-47af-bf0f-eef3ff2e2648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:46.784186 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1351/2697] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:46.788346 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f2cd2-5fed-44ee-8d41-1dc111631960 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:46.799010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f2cd2-5fed-44ee-8d41-1dc111631960 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:46.803556 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd186e-a15c-4da8-a263-cc1119d4f21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:46.804219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd186e-a15c-4da8-a263-cc1119d4f21e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:46.804754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd186e-a15c-4da8-a263-cc1119d4f21e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:46.805022 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd186e-a15c-4da8-a263-cc1119d4f21e admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:46.805730 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd186e-a15c-4da8-a263-cc1119d4f21e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:46.806032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd186e-a15c-4da8-a263-cc1119d4f21e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:46.807693 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd186e-a15c-4da8-a263-cc1119d4f21e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:46.819704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1347/2698] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:12:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:12:46.823743 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832745a-df6e-47af-bf0f-eef3ff2e2648 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:46.833066 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832745a-df6e-47af-bf0f-eef3ff2e2648 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:46.837599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8772299-4409-4581-8eee-0e503089663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:46.838260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8772299-4409-4581-8eee-0e503089663b admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:46.839071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8772299-4409-4581-8eee-0e503089663b admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:46.839346 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8772299-4409-4581-8eee-0e503089663b admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:46.840031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8772299-4409-4581-8eee-0e503089663b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:46.841913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8772299-4409-4581-8eee-0e503089663b admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:46.929925 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1352/2699] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:12:46 2022] POST /identity/v3/projects => generated 386 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:46.934310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd186e-a15c-4da8-a263-cc1119d4f21e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:46.944791 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd186e-a15c-4da8-a263-cc1119d4f21e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:47.003259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be2c47-f533-4dea-8b33-2e049319c51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:47.003943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be2c47-f533-4dea-8b33-2e049319c51c admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:47.004224 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be2c47-f533-4dea-8b33-2e049319c51c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:47.004498 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be2c47-f533-4dea-8b33-2e049319c51c admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:47.005191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be2c47-f533-4dea-8b33-2e049319c51c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:47.007135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be2c47-f533-4dea-8b33-2e049319c51c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:47.077272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1348/2700] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:12:46 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:12:47.081251 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8772299-4409-4581-8eee-0e503089663b admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:47.092236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8772299-4409-4581-8eee-0e503089663b admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:47.096764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db8a35-152d-4390-a21c-f6fd8a9826a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:47.097437 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db8a35-152d-4390-a21c-f6fd8a9826a4 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:47.097717 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db8a35-152d-4390-a21c-f6fd8a9826a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:47.098056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db8a35-152d-4390-a21c-f6fd8a9826a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:47.098759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db8a35-152d-4390-a21c-f6fd8a9826a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:47.100377 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db8a35-152d-4390-a21c-f6fd8a9826a4 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:47.112205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1353/2701] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:47 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:47.116332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be2c47-f533-4dea-8b33-2e049319c51c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:47.126286 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be2c47-f533-4dea-8b33-2e049319c51c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:47.130807 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b62cf-7e02-4773-bf95-6214ac13fa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:47.131486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b62cf-7e02-4773-bf95-6214ac13fa97 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:47.131765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b62cf-7e02-4773-bf95-6214ac13fa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:47.132034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b62cf-7e02-4773-bf95-6214ac13fa97 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:47.132747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b62cf-7e02-4773-bf95-6214ac13fa97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:47.134313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b62cf-7e02-4773-bf95-6214ac13fa97 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:47.144862 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1349/2702] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:47 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:47.148943 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db8a35-152d-4390-a21c-f6fd8a9826a4 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:47.159469 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db8a35-152d-4390-a21c-f6fd8a9826a4 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:47.164009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e66f2c-4cfe-49fb-9eb0-9889f4c12cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:47.164879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e66f2c-4cfe-49fb-9eb0-9889f4c12cf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:47.165159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e66f2c-4cfe-49fb-9eb0-9889f4c12cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:47.165428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e66f2c-4cfe-49fb-9eb0-9889f4c12cf0 admin admin] PATH_INFO: `/v3/projects/f63ac84456f3407e9c06e82aca1d4a74/users/234d261087ba452a871f17f8cfe17cb2/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:47.179840 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e66f2c-4cfe-49fb-9eb0-9889f4c12cf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63ac84456f3407e9c06e82aca1d4a74, user_id=234d261087ba452a871f17f8cfe17cb2, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:47.182505 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e66f2c-4cfe-49fb-9eb0-9889f4c12cf0 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:47.247395 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1354/2703] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:12:47 2022] PUT /identity/v3/projects/f63ac84456f3407e9c06e82aca1d4a74/users/234d261087ba452a871f17f8cfe17cb2/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:47.257716 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d18c5-4638-4213-9c56-88ef7f1f0155 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:47.258034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d18c5-4638-4213-9c56-88ef7f1f0155 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:47.258313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d18c5-4638-4213-9c56-88ef7f1f0155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:47.285718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e50d18c5-4638-4213-9c56-88ef7f1f0155 None None] MFA Rules not processed for user `234d261087ba452a871f17f8cfe17cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:47.286945 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e50d18c5-4638-4213-9c56-88ef7f1f0155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:47.350180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1350/2704] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:47.361513 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e66f2c-4cfe-49fb-9eb0-9889f4c12cf0 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:47.363197 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-75e66f2c-4cfe-49fb-9eb0-9889f4c12cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:47.386127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:47.424763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e66f2c-4cfe-49fb-9eb0-9889f4c12cf0 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:47.428599 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0a0b6-3b46-4931-a890-1483bdf71a1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:47.429295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0a0b6-3b46-4931-a890-1483bdf71a1a service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:47.429832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0a0b6-3b46-4931-a890-1483bdf71a1a service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:47.430103 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0a0b6-3b46-4931-a890-1483bdf71a1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:47.439504 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0a0b6-3b46-4931-a890-1483bdf71a1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:47.440931 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0a0b6-3b46-4931-a890-1483bdf71a1a service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:47.450971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1355/2705] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:12:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:49.746260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d18c5-4638-4213-9c56-88ef7f1f0155 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:49.747696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e50d18c5-4638-4213-9c56-88ef7f1f0155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:49.769763 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:49.840629 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d18c5-4638-4213-9c56-88ef7f1f0155 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:49.845677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b420b9b-b05e-41c7-8bef-f4f95f18783d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:49.846326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b420b9b-b05e-41c7-8bef-f4f95f18783d service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:49.846613 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b420b9b-b05e-41c7-8bef-f4f95f18783d service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:49.846918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b420b9b-b05e-41c7-8bef-f4f95f18783d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:49.848697 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5b420b9b-b05e-41c7-8bef-f4f95f18783d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:49.860661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b420b9b-b05e-41c7-8bef-f4f95f18783d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:49.862306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b420b9b-b05e-41c7-8bef-f4f95f18783d service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:49.873669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1351/2706] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:12:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:50.528392 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0a0b6-3b46-4931-a890-1483bdf71a1a service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:50.590128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0a0b6-3b46-4931-a890-1483bdf71a1a service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:50.594731 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:50.616272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:50.616272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:50.616272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] PATH_INFO: `/v3/users/234d261087ba452a871f17f8cfe17cb2` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:50.616272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=234d261087ba452a871f17f8cfe17cb2)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:50.616272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:50.757986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '234d261087ba452a871f17f8cfe17cb2'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:50.765750 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '234d261087ba452a871f17f8cfe17cb2'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:50.766425 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '234d261087ba452a871f17f8cfe17cb2'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:50.778097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '234d261087ba452a871f17f8cfe17cb2'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:50.824683 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '234d261087ba452a871f17f8cfe17cb2'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:50.830924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1356/2707] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:12:50 2022] DELETE /identity/v3/users/234d261087ba452a871f17f8cfe17cb2 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:50.835571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f8552-8b8e-4fc8-92f9-3b583220838a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:50.836066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f8552-8b8e-4fc8-92f9-3b583220838a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:50.836180 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f8552-8b8e-4fc8-92f9-3b583220838a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:50.853761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-fa0f8552-8b8e-4fc8-92f9-3b583220838a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:50.855189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fa0f8552-8b8e-4fc8-92f9-3b583220838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:50.912922 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1352/2708] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:50.922592 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:50.924011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:50.970312 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:50.981838 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640a43e-f872-4ce5-873b-c4e142522240 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:50.985508 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e5097-5862-423f-9167-823e3b603fee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:50.986781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e5097-5862-423f-9167-823e3b603fee service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:50.986781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e5097-5862-423f-9167-823e3b603fee service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:50.986781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e5097-5862-423f-9167-823e3b603fee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:50.996528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e5097-5862-423f-9167-823e3b603fee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:50.997958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e5097-5862-423f-9167-823e3b603fee service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:51.007669 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1357/2709] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:51.417177 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f8552-8b8e-4fc8-92f9-3b583220838a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:51.419341 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fa0f8552-8b8e-4fc8-92f9-3b583220838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:51.439990 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f8552-8b8e-4fc8-92f9-3b583220838a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:51.446742 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36480e-b5bf-4b06-b73f-f6e88f537936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:51.447672 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36480e-b5bf-4b06-b73f-f6e88f537936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:51.448500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36480e-b5bf-4b06-b73f-f6e88f537936 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:51.448892 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36480e-b5bf-4b06-b73f-f6e88f537936 admin admin] PATH_INFO: `/v3/projects/f63ac84456f3407e9c06e82aca1d4a74` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:51.450153 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36480e-b5bf-4b06-b73f-f6e88f537936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63ac84456f3407e9c06e82aca1d4a74)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:51.452441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36480e-b5bf-4b06-b73f-f6e88f537936 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:51.568765 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ba36480e-b5bf-4b06-b73f-f6e88f537936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63ac84456f3407e9c06e82aca1d4a74', 'request_id': 'req-ba36480e-b5bf-4b06-b73f-f6e88f537936'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:51.642263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ba36480e-b5bf-4b06-b73f-f6e88f537936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63ac84456f3407e9c06e82aca1d4a74', 'request_id': 'req-ba36480e-b5bf-4b06-b73f-f6e88f537936'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:51.649218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ba36480e-b5bf-4b06-b73f-f6e88f537936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63ac84456f3407e9c06e82aca1d4a74', 'request_id': 'req-ba36480e-b5bf-4b06-b73f-f6e88f537936'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:51.661017 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ba36480e-b5bf-4b06-b73f-f6e88f537936 admin admin] The token cache is being invalidate because project f63ac84456f3407e9c06e82aca1d4a74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:51.661349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ba36480e-b5bf-4b06-b73f-f6e88f537936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:51.661993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ba36480e-b5bf-4b06-b73f-f6e88f537936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:51.663559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1353/2710] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:12:51 2022] DELETE /identity/v3/projects/f63ac84456f3407e9c06e82aca1d4a74 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:51.669444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83862a-827a-4844-8df6-45f9e53f7b0a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:51.669928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83862a-827a-4844-8df6-45f9e53f7b0a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:51.670035 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83862a-827a-4844-8df6-45f9e53f7b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:51.689351 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-4e83862a-827a-4844-8df6-45f9e53f7b0a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:51.690544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4e83862a-827a-4844-8df6-45f9e53f7b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:51.752065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1358/2711] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:51.813768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6695a-af56-4ab8-94b0-22b1fd86be9b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:51.814053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6695a-af56-4ab8-94b0-22b1fd86be9b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:51.814334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6695a-af56-4ab8-94b0-22b1fd86be9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:51.833357 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9ca6695a-af56-4ab8-94b0-22b1fd86be9b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:51.834579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9ca6695a-af56-4ab8-94b0-22b1fd86be9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:51.843184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1354/2712] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:51.847528 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83862a-827a-4844-8df6-45f9e53f7b0a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:51.857877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83862a-827a-4844-8df6-45f9e53f7b0a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:51.862539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb180b-f000-493b-acce-f39206a4ae8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:51.863302 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb180b-f000-493b-acce-f39206a4ae8c admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:51.863588 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb180b-f000-493b-acce-f39206a4ae8c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:51.863867 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb180b-f000-493b-acce-f39206a4ae8c admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:51.864586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb180b-f000-493b-acce-f39206a4ae8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:51.864897 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb180b-f000-493b-acce-f39206a4ae8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:51.866544 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb180b-f000-493b-acce-f39206a4ae8c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:51.882872 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1359/2713] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:12:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:12:51.884947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6695a-af56-4ab8-94b0-22b1fd86be9b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:51.891688 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:51.895641 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6695a-af56-4ab8-94b0-22b1fd86be9b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:51.900670 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343009a-86ea-4b83-8b5d-ff24668d6f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:51.904399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343009a-86ea-4b83-8b5d-ff24668d6f8c admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:51.904399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343009a-86ea-4b83-8b5d-ff24668d6f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:51.904399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343009a-86ea-4b83-8b5d-ff24668d6f8c admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:51.904399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343009a-86ea-4b83-8b5d-ff24668d6f8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:51.907050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343009a-86ea-4b83-8b5d-ff24668d6f8c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:51.988242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1355/2714] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:12:51 2022] POST /identity/v3/projects => generated 374 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:12:51.992334 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb180b-f000-493b-acce-f39206a4ae8c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:52.005849 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb180b-f000-493b-acce-f39206a4ae8c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:52.068439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e994b8-d20a-4cd0-b39a-53de2761dbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:52.069305 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e994b8-d20a-4cd0-b39a-53de2761dbb8 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:52.070060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e994b8-d20a-4cd0-b39a-53de2761dbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:52.070060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e994b8-d20a-4cd0-b39a-53de2761dbb8 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:52.070647 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e994b8-d20a-4cd0-b39a-53de2761dbb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:52.072610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e994b8-d20a-4cd0-b39a-53de2761dbb8 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:52.157457 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1360/2715] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:12:51 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:52.161696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343009a-86ea-4b83-8b5d-ff24668d6f8c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:52.172212 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343009a-86ea-4b83-8b5d-ff24668d6f8c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:52.186441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce30ef-f1fa-4523-a754-2bbfd92af588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:52.187156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce30ef-f1fa-4523-a754-2bbfd92af588 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:52.187432 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce30ef-f1fa-4523-a754-2bbfd92af588 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:52.187702 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce30ef-f1fa-4523-a754-2bbfd92af588 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:52.188427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce30ef-f1fa-4523-a754-2bbfd92af588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:52.190148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce30ef-f1fa-4523-a754-2bbfd92af588 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:52.201213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1356/2716] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:52 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:52.205093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e994b8-d20a-4cd0-b39a-53de2761dbb8 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:52.215729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e994b8-d20a-4cd0-b39a-53de2761dbb8 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:52.220272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6bc3d-d5de-4835-bc98-1c96c7beca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:52.220924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6bc3d-d5de-4835-bc98-1c96c7beca70 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:52.221202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6bc3d-d5de-4835-bc98-1c96c7beca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:52.221473 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6bc3d-d5de-4835-bc98-1c96c7beca70 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:52.222188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6bc3d-d5de-4835-bc98-1c96c7beca70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:52.223802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6bc3d-d5de-4835-bc98-1c96c7beca70 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:52.233767 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1361/2717] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:52 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:52.237919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce30ef-f1fa-4523-a754-2bbfd92af588 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:52.250217 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce30ef-f1fa-4523-a754-2bbfd92af588 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:52.254865 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2f8a0-f81b-4778-b34d-c37c9701bb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:52.255746 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2f8a0-f81b-4778-b34d-c37c9701bb7d admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:52.256032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2f8a0-f81b-4778-b34d-c37c9701bb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:52.256309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2f8a0-f81b-4778-b34d-c37c9701bb7d admin admin] PATH_INFO: `/v3/projects/a50fbd483e2e4d8886ae5d6341af7dcb/users/cdc1042ea3b54768ab87e1f66e74ec1c/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:52.271682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2f8a0-f81b-4778-b34d-c37c9701bb7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a50fbd483e2e4d8886ae5d6341af7dcb, user_id=cdc1042ea3b54768ab87e1f66e74ec1c, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:52.274323 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2f8a0-f81b-4778-b34d-c37c9701bb7d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:52.345563 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1357/2718] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:12:52 2022] PUT /identity/v3/projects/a50fbd483e2e4d8886ae5d6341af7dcb/users/cdc1042ea3b54768ab87e1f66e74ec1c/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:52.350552 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28390e-4e6d-43a3-906b-9c5c97970aca None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:52.350854 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28390e-4e6d-43a3-906b-9c5c97970aca None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:52.351191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28390e-4e6d-43a3-906b-9c5c97970aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:52.368631 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1b28390e-4e6d-43a3-906b-9c5c97970aca None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:52.369936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1b28390e-4e6d-43a3-906b-9c5c97970aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:52.414540 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:52.428333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1362/2719] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:52.438214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2f8a0-f81b-4778-b34d-c37c9701bb7d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:52.440005 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fbd2f8a0-f81b-4778-b34d-c37c9701bb7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:52.496988 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2f8a0-f81b-4778-b34d-c37c9701bb7d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:52.500674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f908e-b5c9-47e3-b66a-bb9a9cf62f79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:52.501313 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f908e-b5c9-47e3-b66a-bb9a9cf62f79 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:52.501593 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f908e-b5c9-47e3-b66a-bb9a9cf62f79 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:52.501863 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f908e-b5c9-47e3-b66a-bb9a9cf62f79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:52.511301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f908e-b5c9-47e3-b66a-bb9a9cf62f79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:52.512757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f908e-b5c9-47e3-b66a-bb9a9cf62f79 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:52.522026 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1358/2720] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:56.893499 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f96f64-38ec-4a45-a622-ba325774e44b None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:56.894021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f96f64-38ec-4a45-a622-ba325774e44b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:56.894021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f96f64-38ec-4a45-a622-ba325774e44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:56.911982 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c3f96f64-38ec-4a45-a622-ba325774e44b None None] MFA Rules not processed for user `e7e74d9bfa6144ae8ad7e3156fb42e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:56.913172 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c3f96f64-38ec-4a45-a622-ba325774e44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:56.971816 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1363/2721] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:58.473041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d7ef0-0005-473d-b559-a95c8f85c4f3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:58.473590 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d7ef0-0005-473d-b559-a95c8f85c4f3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:58.473590 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d7ef0-0005-473d-b559-a95c8f85c4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:58.503540 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9b0d7ef0-0005-473d-b559-a95c8f85c4f3 None None] MFA Rules not processed for user `cdc1042ea3b54768ab87e1f66e74ec1c`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:58.504858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9b0d7ef0-0005-473d-b559-a95c8f85c4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:58.581961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1359/2722] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:12:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:58.586141 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f96f64-38ec-4a45-a622-ba325774e44b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:58.597896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f96f64-38ec-4a45-a622-ba325774e44b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:58.602740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4118f39-441c-41d1-a21f-871a1ffe6286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:58.609877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4118f39-441c-41d1-a21f-871a1ffe6286 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:58.609877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4118f39-441c-41d1-a21f-871a1ffe6286 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:58.609877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4118f39-441c-41d1-a21f-871a1ffe6286 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:58.609877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4118f39-441c-41d1-a21f-871a1ffe6286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:58.609877 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4118f39-441c-41d1-a21f-871a1ffe6286 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:58.723371 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1364/2723] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:12:58 2022] POST /identity/v3/projects => generated 372 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:58.727528 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d7ef0-0005-473d-b559-a95c8f85c4f3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:58.738702 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d7ef0-0005-473d-b559-a95c8f85c4f3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:58.784223 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:58.804920 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6d020-b888-4638-99bf-e80490f2b36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:58.805601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6d020-b888-4638-99bf-e80490f2b36f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:58.806036 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6d020-b888-4638-99bf-e80490f2b36f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:58.806310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6d020-b888-4638-99bf-e80490f2b36f admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:58.807144 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6d020-b888-4638-99bf-e80490f2b36f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:58.809214 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6d020-b888-4638-99bf-e80490f2b36f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:58.887604 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1360/2724] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:12:58 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:58.891538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4118f39-441c-41d1-a21f-871a1ffe6286 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:58.902687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4118f39-441c-41d1-a21f-871a1ffe6286 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:58.907382 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a944638-8cda-4e97-b752-b6c7500b185c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:58.908055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a944638-8cda-4e97-b752-b6c7500b185c admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:58.908340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a944638-8cda-4e97-b752-b6c7500b185c admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:58.908616 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a944638-8cda-4e97-b752-b6c7500b185c admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:58.909337 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a944638-8cda-4e97-b752-b6c7500b185c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:58.910960 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a944638-8cda-4e97-b752-b6c7500b185c admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:58.922262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1365/2725] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:58 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:58.926464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6d020-b888-4638-99bf-e80490f2b36f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:58.936982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6d020-b888-4638-99bf-e80490f2b36f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:58.942347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fba4cc-0196-4909-9c4b-f00e1276f331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:58.943425 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fba4cc-0196-4909-9c4b-f00e1276f331 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:58.943713 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fba4cc-0196-4909-9c4b-f00e1276f331 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:58.943992 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fba4cc-0196-4909-9c4b-f00e1276f331 admin admin] PATH_INFO: `/v3/projects/16d607c437d74fd4a328d6a33ef9fafe/users/233d5c99300642959bfa85a9e44f232e/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:58.959403 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fba4cc-0196-4909-9c4b-f00e1276f331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d607c437d74fd4a328d6a33ef9fafe, user_id=233d5c99300642959bfa85a9e44f232e, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:58.962105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fba4cc-0196-4909-9c4b-f00e1276f331 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:59.036486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1361/2726] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:12:58 2022] PUT /identity/v3/projects/16d607c437d74fd4a328d6a33ef9fafe/users/233d5c99300642959bfa85a9e44f232e/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:59.040524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a944638-8cda-4e97-b752-b6c7500b185c admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:59.053991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a944638-8cda-4e97-b752-b6c7500b185c admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:59.125653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5f849-c408-478d-89c6-0ccf67638ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:59.126391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5f849-c408-478d-89c6-0ccf67638ebb admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:59.126670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5f849-c408-478d-89c6-0ccf67638ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:59.126986 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5f849-c408-478d-89c6-0ccf67638ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:59.127713 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5f849-c408-478d-89c6-0ccf67638ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:59.129298 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5f849-c408-478d-89c6-0ccf67638ebb admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:59.144392 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1366/2727] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:12:59 2022] GET /identity/v3/roles => generated 1427 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:59.145362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fba4cc-0196-4909-9c4b-f00e1276f331 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:59.157386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fba4cc-0196-4909-9c4b-f00e1276f331 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:59.162174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979287b-774a-443a-a607-97956779fa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:59.163096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979287b-774a-443a-a607-97956779fa99 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:59.163375 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979287b-774a-443a-a607-97956779fa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:59.163647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979287b-774a-443a-a607-97956779fa99 admin admin] PATH_INFO: `/v3/projects/16d607c437d74fd4a328d6a33ef9fafe/users/233d5c99300642959bfa85a9e44f232e/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:59.165586 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979287b-774a-443a-a607-97956779fa99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d607c437d74fd4a328d6a33ef9fafe, user_id=233d5c99300642959bfa85a9e44f232e, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:59.169034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979287b-774a-443a-a607-97956779fa99 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:59.180406 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1362/2728] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:12:59 2022] PUT /identity/v3/projects/16d607c437d74fd4a328d6a33ef9fafe/users/233d5c99300642959bfa85a9e44f232e/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:13:04.886358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34704efc-2ac9-4aa7-9c4c-5be64f8a7953 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:04.886873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34704efc-2ac9-4aa7-9c4c-5be64f8a7953 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:04.887090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34704efc-2ac9-4aa7-9c4c-5be64f8a7953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:04.919091 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-34704efc-2ac9-4aa7-9c4c-5be64f8a7953 None None] MFA Rules not processed for user `233d5c99300642959bfa85a9e44f232e`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:04.920436 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-34704efc-2ac9-4aa7-9c4c-5be64f8a7953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:04.986505 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:05.008966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1367/2729] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:13:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:13:05.023369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979287b-774a-443a-a607-97956779fa99 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:05.025183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d979287b-774a-443a-a607-97956779fa99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:05.109683 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979287b-774a-443a-a607-97956779fa99 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:05.115464 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d948e3-47ad-435f-a75e-1327c69e9bff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:05.116134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d948e3-47ad-435f-a75e-1327c69e9bff service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:05.116412 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d948e3-47ad-435f-a75e-1327c69e9bff service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:05.116687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d948e3-47ad-435f-a75e-1327c69e9bff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:05.180517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d948e3-47ad-435f-a75e-1327c69e9bff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:05.183340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d948e3-47ad-435f-a75e-1327c69e9bff service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:05.199359 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1363/2730] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:13:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:13:08.291256 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e3f3a-5992-4487-93d9-5b1b909a41c8 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:08.291878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e3f3a-5992-4487-93d9-5b1b909a41c8 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:08.292147 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e3f3a-5992-4487-93d9-5b1b909a41c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:08.311356 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b39e3f3a-5992-4487-93d9-5b1b909a41c8 None None] MFA Rules not processed for user `cdc1042ea3b54768ab87e1f66e74ec1c`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:08.312576 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b39e3f3a-5992-4487-93d9-5b1b909a41c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:08.326596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1368/2731] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:13:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:13:15.189558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:18.305478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=10 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:24.910920 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d948e3-47ad-435f-a75e-1327c69e9bff service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:24.912512 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-78d948e3-47ad-435f-a75e-1327c69e9bff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:24.928539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d948e3-47ad-435f-a75e-1327c69e9bff service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:24.934370 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d030a76-4fb2-4e26-8466-41bf783d427b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:24.935597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d030a76-4fb2-4e26-8466-41bf783d427b service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:24.935597 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d030a76-4fb2-4e26-8466-41bf783d427b service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:24.936202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d030a76-4fb2-4e26-8466-41bf783d427b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:24.937515 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9d030a76-4fb2-4e26-8466-41bf783d427b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:24.950038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d030a76-4fb2-4e26-8466-41bf783d427b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:24.951661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d030a76-4fb2-4e26-8466-41bf783d427b service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:24.963045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1364/2732] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:13:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:13:34.958998 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:51.393023 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e3f3a-5992-4487-93d9-5b1b909a41c8 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:51.395701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b39e3f3a-5992-4487-93d9-5b1b909a41c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:51.461455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e3f3a-5992-4487-93d9-5b1b909a41c8 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:51.465006 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2347c1d-dbfa-4a3a-82a1-1a543879d0c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:51.465670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2347c1d-dbfa-4a3a-82a1-1a543879d0c4 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:51.465954 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2347c1d-dbfa-4a3a-82a1-1a543879d0c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:51.466238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2347c1d-dbfa-4a3a-82a1-1a543879d0c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:51.476110 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2347c1d-dbfa-4a3a-82a1-1a543879d0c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:51.477563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2347c1d-dbfa-4a3a-82a1-1a543879d0c4 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:51.487522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1369/2733] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:13:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:14:01.483058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=26 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:14:41.851650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d030a76-4fb2-4e26-8466-41bf783d427b service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:14:41.861189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d030a76-4fb2-4e26-8466-41bf783d427b service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:14:41.865185 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0719774-df01-4e2f-af6c-75e58e23e3cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:14:41.865576 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0719774-df01-4e2f-af6c-75e58e23e3cd service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:14:41.865856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0719774-df01-4e2f-af6c-75e58e23e3cd service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:14:41.866131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0719774-df01-4e2f-af6c-75e58e23e3cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:14:41.867940 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f0719774-df01-4e2f-af6c-75e58e23e3cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:14:41.882698 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0719774-df01-4e2f-af6c-75e58e23e3cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:14:41.884195 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0719774-df01-4e2f-af6c-75e58e23e3cd service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:14:41.895737 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1365/2734] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:14:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:14:45.414310 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2347c1d-dbfa-4a3a-82a1-1a543879d0c4 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:14:45.423227 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2347c1d-dbfa-4a3a-82a1-1a543879d0c4 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:14:45.427097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed30b2-0413-47d8-8c09-2280044c682b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:14:45.427739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed30b2-0413-47d8-8c09-2280044c682b service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:14:45.428021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed30b2-0413-47d8-8c09-2280044c682b service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:14:45.428295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed30b2-0413-47d8-8c09-2280044c682b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:14:45.430027 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4bed30b2-0413-47d8-8c09-2280044c682b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:14:45.443665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed30b2-0413-47d8-8c09-2280044c682b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:14:45.445088 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed30b2-0413-47d8-8c09-2280044c682b service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:14:45.455992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1370/2735] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:14:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:14:51.890200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:14:55.451182 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=6 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:14.588994 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0719774-df01-4e2f-af6c-75e58e23e3cd service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:14.590382 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f0719774-df01-4e2f-af6c-75e58e23e3cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:14.645811 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0719774-df01-4e2f-af6c-75e58e23e3cd service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:14.649640 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81c242-71c9-47c7-ae73-ded6a146c2c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:14.650287 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81c242-71c9-47c7-ae73-ded6a146c2c1 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:14.650561 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81c242-71c9-47c7-ae73-ded6a146c2c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:14.650848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81c242-71c9-47c7-ae73-ded6a146c2c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:14.652621 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6e81c242-71c9-47c7-ae73-ded6a146c2c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:14.711502 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81c242-71c9-47c7-ae73-ded6a146c2c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:14.712931 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81c242-71c9-47c7-ae73-ded6a146c2c1 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:14.721918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1366/2736] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:15:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:24.717768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=48 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:27.314199 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71e43e-c91d-4fa7-9224-0d2230f4bac1 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:27.314709 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71e43e-c91d-4fa7-9224-0d2230f4bac1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:27.314846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71e43e-c91d-4fa7-9224-0d2230f4bac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:27.332806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-3b71e43e-c91d-4fa7-9224-0d2230f4bac1 None None] MFA Rules not processed for user `e7e74d9bfa6144ae8ad7e3156fb42e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:27.334033 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3b71e43e-c91d-4fa7-9224-0d2230f4bac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:27.341777 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1371/2737] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:15:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:27.654092 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81c242-71c9-47c7-ae73-ded6a146c2c1 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:27.663348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81c242-71c9-47c7-ae73-ded6a146c2c1 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:27.667559 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d42bfd-b84c-40b0-974c-e89f906bca43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:27.668198 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d42bfd-b84c-40b0-974c-e89f906bca43 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:27.668484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d42bfd-b84c-40b0-974c-e89f906bca43 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:27.668761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d42bfd-b84c-40b0-974c-e89f906bca43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:27.670530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-71d42bfd-b84c-40b0-974c-e89f906bca43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:27.680045 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d42bfd-b84c-40b0-974c-e89f906bca43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:27.682572 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d42bfd-b84c-40b0-974c-e89f906bca43 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:27.690591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1367/2738] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:15:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:28.381330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71e43e-c91d-4fa7-9224-0d2230f4bac1 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:28.382873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-3b71e43e-c91d-4fa7-9224-0d2230f4bac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:28.454229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71e43e-c91d-4fa7-9224-0d2230f4bac1 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:28.459069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887c990-6c55-4a24-9b0a-6d4270c9acef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:28.459711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887c990-6c55-4a24-9b0a-6d4270c9acef service placement] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:28.459987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887c990-6c55-4a24-9b0a-6d4270c9acef service placement] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:28.460255 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887c990-6c55-4a24-9b0a-6d4270c9acef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:28.462563 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c887c990-6c55-4a24-9b0a-6d4270c9acef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:28.473521 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887c990-6c55-4a24-9b0a-6d4270c9acef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:28.475124 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887c990-6c55-4a24-9b0a-6d4270c9acef service placement] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:28.486426 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1372/2739] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:15:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:31.100219 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d42bfd-b84c-40b0-974c-e89f906bca43 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:31.112145 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d42bfd-b84c-40b0-974c-e89f906bca43 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:31.115856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43684ea5-a0e3-40b7-beed-4bed2fdc04bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:31.116496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43684ea5-a0e3-40b7-beed-4bed2fdc04bc service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:31.116774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43684ea5-a0e3-40b7-beed-4bed2fdc04bc service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:31.117044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43684ea5-a0e3-40b7-beed-4bed2fdc04bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:31.119474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-43684ea5-a0e3-40b7-beed-4bed2fdc04bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:31.129200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43684ea5-a0e3-40b7-beed-4bed2fdc04bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:31.133256 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43684ea5-a0e3-40b7-beed-4bed2fdc04bc service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:31.138872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1368/2740] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:15:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:31.922610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887c990-6c55-4a24-9b0a-6d4270c9acef service placement] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:31.937367 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887c990-6c55-4a24-9b0a-6d4270c9acef service placement] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:31.943602 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771cacb-bf29-4af3-afec-700b32d2dbaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:31.944275 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771cacb-bf29-4af3-afec-700b32d2dbaf service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:31.944560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771cacb-bf29-4af3-afec-700b32d2dbaf service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:31.944993 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771cacb-bf29-4af3-afec-700b32d2dbaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:31.947447 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9771cacb-bf29-4af3-afec-700b32d2dbaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:31.960610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771cacb-bf29-4af3-afec-700b32d2dbaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:31.962333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771cacb-bf29-4af3-afec-700b32d2dbaf service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:31.974031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1373/2741] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:15:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:41.135620 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=12 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:41.969011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=41 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:46.199932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43684ea5-a0e3-40b7-beed-4bed2fdc04bc service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:46.208777 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43684ea5-a0e3-40b7-beed-4bed2fdc04bc service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:46.212449 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be9957-1458-4ce9-b484-f8f3ceffa400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:46.213085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be9957-1458-4ce9-b484-f8f3ceffa400 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:46.213517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be9957-1458-4ce9-b484-f8f3ceffa400 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:46.213823 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be9957-1458-4ce9-b484-f8f3ceffa400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:46.215604 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d0be9957-1458-4ce9-b484-f8f3ceffa400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:46.278626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be9957-1458-4ce9-b484-f8f3ceffa400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:46.280094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be9957-1458-4ce9-b484-f8f3ceffa400 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:46.290213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1369/2742] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:15:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:15:56.285682 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=30 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:16.413038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cc9f8-8a31-4b3e-95f0-73bc77af91ac None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:16.413545 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cc9f8-8a31-4b3e-95f0-73bc77af91ac None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:16.413652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cc9f8-8a31-4b3e-95f0-73bc77af91ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:16.561038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-465cc9f8-8a31-4b3e-95f0-73bc77af91ac None None] MFA Rules not processed for user `7df2a24143654445bcb96df81e960c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:16.562248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-465cc9f8-8a31-4b3e-95f0-73bc77af91ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:16.616811 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1374/2743] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:16:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:16.628735 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be9957-1458-4ce9-b484-f8f3ceffa400 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:16.640696 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be9957-1458-4ce9-b484-f8f3ceffa400 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:16.647020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2906640-dcf6-44c2-ba42-da3fef629c9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:16.647687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2906640-dcf6-44c2-ba42-da3fef629c9b service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:16.647961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2906640-dcf6-44c2-ba42-da3fef629c9b service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:16.648200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2906640-dcf6-44c2-ba42-da3fef629c9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:16.649929 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f2906640-dcf6-44c2-ba42-da3fef629c9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:16.660365 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2906640-dcf6-44c2-ba42-da3fef629c9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:16.661939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2906640-dcf6-44c2-ba42-da3fef629c9b service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:16.670745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1370/2744] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:16:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:25.760763 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cc9f8-8a31-4b3e-95f0-73bc77af91ac None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:25.770806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cc9f8-8a31-4b3e-95f0-73bc77af91ac None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:25.774950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549edcf3-c9db-4c60-a5a5-03ff6835d13a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:25.775662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549edcf3-c9db-4c60-a5a5-03ff6835d13a service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:25.775921 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549edcf3-c9db-4c60-a5a5-03ff6835d13a service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:25.776400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549edcf3-c9db-4c60-a5a5-03ff6835d13a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:25.778703 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-549edcf3-c9db-4c60-a5a5-03ff6835d13a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:25.794057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549edcf3-c9db-4c60-a5a5-03ff6835d13a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:25.795530 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549edcf3-c9db-4c60-a5a5-03ff6835d13a service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:25.808362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1375/2745] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:16:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:26.666731 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:27.151799 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada86827-3659-42bc-98fd-62b12bb61570 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:27.152102 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada86827-3659-42bc-98fd-62b12bb61570 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:27.152356 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada86827-3659-42bc-98fd-62b12bb61570 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:27.153826 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1371/2746] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 18:16:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 18:16:27.163283 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a55d19-f2f0-47dd-9a97-0edc4e710153 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:27.163864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a55d19-f2f0-47dd-9a97-0edc4e710153 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:27.163864 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a55d19-f2f0-47dd-9a97-0edc4e710153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:27.216439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-49a55d19-f2f0-47dd-9a97-0edc4e710153 None None] MFA Rules not processed for user `08709b462300437598c8db331e5ad20f`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:27.217828 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-49a55d19-f2f0-47dd-9a97-0edc4e710153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:27.226851 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1376/2747] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 18:16:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:27.243974 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada86827-3659-42bc-98fd-62b12bb61570 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:27.263246 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada86827-3659-42bc-98fd-62b12bb61570 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:27.270282 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be252446-19fe-4507-be6e-73246934a335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:27.271310 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be252446-19fe-4507-be6e-73246934a335 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:27.271665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be252446-19fe-4507-be6e-73246934a335 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:27.272037 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be252446-19fe-4507-be6e-73246934a335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:27.280086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be252446-19fe-4507-be6e-73246934a335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:27.281585 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be252446-19fe-4507-be6e-73246934a335 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:27.290296 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1372/2748] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:16:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:27.412481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c3fa0-c7ef-4d6e-a6f5-ab2243303c50 None None] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:27.413138 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c3fa0-c7ef-4d6e-a6f5-ab2243303c50 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:27.413652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c3fa0-c7ef-4d6e-a6f5-ab2243303c50 None None] PATH_INFO: `/` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:27.414797 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1377/2749] 10.222.0.121 () {54 vars in 926 bytes} [Thu May 26 18:16:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) May 26 18:16:27.420722 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe6eb9-72e4-423d-926f-dc50dc9a2ea5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:27.421008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe6eb9-72e4-423d-926f-dc50dc9a2ea5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:27.421285 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe6eb9-72e4-423d-926f-dc50dc9a2ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:27.459290 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-cbbe6eb9-72e4-423d-926f-dc50dc9a2ea5 None None] MFA Rules not processed for user `c0889a69a018452d8de1924072739d17`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:27.460496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cbbe6eb9-72e4-423d-926f-dc50dc9a2ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:27.469992 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1373/2750] 10.222.0.121 () {58 vars in 1023 bytes} [Thu May 26 18:16:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:27.480710 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c3fa0-c7ef-4d6e-a6f5-ab2243303c50 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:27.491906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c3fa0-c7ef-4d6e-a6f5-ab2243303c50 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:27.495313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699efd14-d38d-41fd-9f97-695848bffbc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:27.495939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699efd14-d38d-41fd-9f97-695848bffbc9 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:27.496217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699efd14-d38d-41fd-9f97-695848bffbc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:27.496495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699efd14-d38d-41fd-9f97-695848bffbc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:27.506938 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699efd14-d38d-41fd-9f97-695848bffbc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:27.508315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699efd14-d38d-41fd-9f97-695848bffbc9 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:27.518810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1378/2751] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:16:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:37.456969 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=15 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:37.514114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=50 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:41.265891 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe6eb9-72e4-423d-926f-dc50dc9a2ea5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:41.277952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe6eb9-72e4-423d-926f-dc50dc9a2ea5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:41.283551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac82b0-a026-452b-b1b7-2acd2d9eddb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:41.284317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac82b0-a026-452b-b1b7-2acd2d9eddb1 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:41.286217 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac82b0-a026-452b-b1b7-2acd2d9eddb1 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:41.286217 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac82b0-a026-452b-b1b7-2acd2d9eddb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:41.286594 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-74ac82b0-a026-452b-b1b7-2acd2d9eddb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:41.298933 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac82b0-a026-452b-b1b7-2acd2d9eddb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:41.301123 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac82b0-a026-452b-b1b7-2acd2d9eddb1 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:41.312522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1374/2752] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:16:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:51.307600 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:56.321771 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699efd14-d38d-41fd-9f97-695848bffbc9 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:56.333010 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699efd14-d38d-41fd-9f97-695848bffbc9 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:56.337880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f5a69-0f81-4a0b-abc5-aef749ba7912 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:56.341764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f5a69-0f81-4a0b-abc5-aef749ba7912 service placement] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:56.341764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f5a69-0f81-4a0b-abc5-aef749ba7912 service placement] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:56.341764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f5a69-0f81-4a0b-abc5-aef749ba7912 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:56.341764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-863f5a69-0f81-4a0b-abc5-aef749ba7912 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:56.353882 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f5a69-0f81-4a0b-abc5-aef749ba7912 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:56.355506 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f5a69-0f81-4a0b-abc5-aef749ba7912 service placement] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:56.370204 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1379/2753] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:17:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:18:06.361670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=6 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:08.648767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac82b0-a026-452b-b1b7-2acd2d9eddb1 service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:08.658186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac82b0-a026-452b-b1b7-2acd2d9eddb1 service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:08.661879 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15027b9c-1c54-4c00-91f2-9e2c82415c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:08.662522 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027b9c-1c54-4c00-91f2-9e2c82415c24 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:08.662803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027b9c-1c54-4c00-91f2-9e2c82415c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:08.663114 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027b9c-1c54-4c00-91f2-9e2c82415c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:08.664914 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-15027b9c-1c54-4c00-91f2-9e2c82415c24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:08.728636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15027b9c-1c54-4c00-91f2-9e2c82415c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:08.729982 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15027b9c-1c54-4c00-91f2-9e2c82415c24 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:08.740286 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1375/2754] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:12.294920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f5a69-0f81-4a0b-abc5-aef749ba7912 service placement] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:12.296805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-863f5a69-0f81-4a0b-abc5-aef749ba7912 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:12.309822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f5a69-0f81-4a0b-abc5-aef749ba7912 service placement] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:12.316707 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4763d-56db-4493-a35d-71b99119a693 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:12.317551 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4763d-56db-4493-a35d-71b99119a693 service placement] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:12.317846 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4763d-56db-4493-a35d-71b99119a693 service placement] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:12.318121 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4763d-56db-4493-a35d-71b99119a693 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:12.320002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-48d4763d-56db-4493-a35d-71b99119a693 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:12.333104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4763d-56db-4493-a35d-71b99119a693 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:12.334868 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4763d-56db-4493-a35d-71b99119a693 service placement] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:12.345246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1380/2755] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:16.091832 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15027b9c-1c54-4c00-91f2-9e2c82415c24 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:16.108254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15027b9c-1c54-4c00-91f2-9e2c82415c24 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:16.114151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2043d-6068-4800-864e-efeb70236916 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:16.114814 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2043d-6068-4800-864e-efeb70236916 service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:16.115139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2043d-6068-4800-864e-efeb70236916 service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:16.115415 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2043d-6068-4800-864e-efeb70236916 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:16.117386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-50f2043d-6068-4800-864e-efeb70236916 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:16.129657 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2043d-6068-4800-864e-efeb70236916 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:16.131287 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2043d-6068-4800-864e-efeb70236916 service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:16.142335 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1376/2756] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:16.235558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4763d-56db-4493-a35d-71b99119a693 service placement] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:16.237104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-48d4763d-56db-4493-a35d-71b99119a693 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:16.251852 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4763d-56db-4493-a35d-71b99119a693 service placement] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:16.256390 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:16.257261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:16.257542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:16.257806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] PATH_INFO: `/v3/users/e7e74d9bfa6144ae8ad7e3156fb42e6e` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:16.259060 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e74d9bfa6144ae8ad7e3156fb42e6e)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:16.260966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:16.544806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e74d9bfa6144ae8ad7e3156fb42e6e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:16.552485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e74d9bfa6144ae8ad7e3156fb42e6e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:16.553210 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e74d9bfa6144ae8ad7e3156fb42e6e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:16.566011 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e74d9bfa6144ae8ad7e3156fb42e6e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:16.671418 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e74d9bfa6144ae8ad7e3156fb42e6e'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:16.678054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1381/2757] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:18:16 2022] DELETE /identity/v3/users/e7e74d9bfa6144ae8ad7e3156fb42e6e => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:16.947596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2043d-6068-4800-864e-efeb70236916 service placement] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:16.961215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2043d-6068-4800-864e-efeb70236916 service placement] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:17.022376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:17.023300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:17.023582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:17.023858 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] PATH_INFO: `/v3/projects/7daf2a694ab840949d8cf403c2573bda` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:17.025075 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7daf2a694ab840949d8cf403c2573bda)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:17.027112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:17.152752 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7daf2a694ab840949d8cf403c2573bda', 'request_id': 'req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.218480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7daf2a694ab840949d8cf403c2573bda', 'request_id': 'req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.226189 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7daf2a694ab840949d8cf403c2573bda', 'request_id': 'req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.236662 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] The token cache is being invalidate because project 7daf2a694ab840949d8cf403c2573bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:17.236942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.237580 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.239167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1377/2758] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:18:16 2022] DELETE /identity/v3/projects/7daf2a694ab840949d8cf403c2573bda => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:17.243160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:17.244562 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:17.257848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3932edf-a673-4794-8e6a-3ba7f4e0c80e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:17.317049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:17.317922 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:17.318202 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:17.318475 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] PATH_INFO: `/v3/users/2b82ed5b3a2c408a8f8623729062baec` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:17.319685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b82ed5b3a2c408a8f8623729062baec)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:17.321656 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:17.536481 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b82ed5b3a2c408a8f8623729062baec'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.543383 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b82ed5b3a2c408a8f8623729062baec'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.544015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b82ed5b3a2c408a8f8623729062baec'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.548322 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:17.556462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b82ed5b3a2c408a8f8623729062baec'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.734897 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b82ed5b3a2c408a8f8623729062baec'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:17.742040 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1382/2759] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:18:17 2022] DELETE /identity/v3/users/2b82ed5b3a2c408a8f8623729062baec => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:18.119319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:18.120814 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:18.134886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c1c87-7f06-4e0b-9b50-9de9ff89e76d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:18.146165 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:18.200611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:18.201483 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:18.201768 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:18.202043 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] PATH_INFO: `/v3/projects/cfc02810c21b45ed8608a00f56ff9fe9` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:18.203256 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc02810c21b45ed8608a00f56ff9fe9)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:18.205467 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:18.323592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc02810c21b45ed8608a00f56ff9fe9', 'request_id': 'req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:18.392120 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc02810c21b45ed8608a00f56ff9fe9', 'request_id': 'req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:18.399393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc02810c21b45ed8608a00f56ff9fe9', 'request_id': 'req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:18.410233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] The token cache is being invalidate because project cfc02810c21b45ed8608a00f56ff9fe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:18.410518 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:18.411190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:18.412859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1378/2760] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:18:18 2022] DELETE /identity/v3/projects/cfc02810c21b45ed8608a00f56ff9fe9 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:23.762914 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:23.764462 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:23.823342 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb179869-c2f4-46e8-b62b-206e2a299210 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:23.876278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d690e-0a62-47d0-9bec-3e77d35a9221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:23.877201 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d690e-0a62-47d0-9bec-3e77d35a9221 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:23.877687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d690e-0a62-47d0-9bec-3e77d35a9221 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:23.877955 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d690e-0a62-47d0-9bec-3e77d35a9221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:23.880696 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5c4d690e-0a62-47d0-9bec-3e77d35a9221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:23.961245 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d690e-0a62-47d0-9bec-3e77d35a9221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:23.962725 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d690e-0a62-47d0-9bec-3e77d35a9221 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:23.973349 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1383/2761] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:28.734947 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:30.608229 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:30.609777 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:30.623148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc34fe-f32a-49cd-a4d9-6124b75d2e37 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:30.626740 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126fc5-1b82-45ac-8187-d8bd03c12696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:30.627433 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126fc5-1b82-45ac-8187-d8bd03c12696 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:30.627715 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126fc5-1b82-45ac-8187-d8bd03c12696 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:30.627991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126fc5-1b82-45ac-8187-d8bd03c12696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:30.630445 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6a126fc5-1b82-45ac-8187-d8bd03c12696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:30.696569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a126fc5-1b82-45ac-8187-d8bd03c12696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:30.698131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a126fc5-1b82-45ac-8187-d8bd03c12696 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:30.708496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1379/2762] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:30.765192 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d690e-0a62-47d0-9bec-3e77d35a9221 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:30.766639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5c4d690e-0a62-47d0-9bec-3e77d35a9221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:30.778187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d690e-0a62-47d0-9bec-3e77d35a9221 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:30.781975 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffa308-994f-40b1-8f42-f7af6f29672a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:30.782701 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffa308-994f-40b1-8f42-f7af6f29672a service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:30.783108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffa308-994f-40b1-8f42-f7af6f29672a service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:30.783453 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffa308-994f-40b1-8f42-f7af6f29672a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:30.785301 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-52ffa308-994f-40b1-8f42-f7af6f29672a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:30.802093 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffa308-994f-40b1-8f42-f7af6f29672a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:30.804225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffa308-994f-40b1-8f42-f7af6f29672a service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:30.814565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1384/2763] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:34.603952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126fc5-1b82-45ac-8187-d8bd03c12696 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:34.605939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6a126fc5-1b82-45ac-8187-d8bd03c12696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:34.618919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126fc5-1b82-45ac-8187-d8bd03c12696 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:34.626318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027a674-8e2d-45a5-ae8e-08caf7594e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:34.627366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027a674-8e2d-45a5-ae8e-08caf7594e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:34.627750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027a674-8e2d-45a5-ae8e-08caf7594e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:34.628125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027a674-8e2d-45a5-ae8e-08caf7594e2c admin admin] PATH_INFO: `/v3/users/7df2a24143654445bcb96df81e960c9d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:34.629656 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027a674-8e2d-45a5-ae8e-08caf7594e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7df2a24143654445bcb96df81e960c9d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:34.639614 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027a674-8e2d-45a5-ae8e-08caf7594e2c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:34.864732 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9027a674-8e2d-45a5-ae8e-08caf7594e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7df2a24143654445bcb96df81e960c9d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:34.873376 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9027a674-8e2d-45a5-ae8e-08caf7594e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7df2a24143654445bcb96df81e960c9d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:34.874200 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9027a674-8e2d-45a5-ae8e-08caf7594e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7df2a24143654445bcb96df81e960c9d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:34.888480 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9027a674-8e2d-45a5-ae8e-08caf7594e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7df2a24143654445bcb96df81e960c9d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:34.991363 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9027a674-8e2d-45a5-ae8e-08caf7594e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7df2a24143654445bcb96df81e960c9d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:34.998068 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1380/2764] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:18:34 2022] DELETE /identity/v3/users/7df2a24143654445bcb96df81e960c9d => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:35.301759 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffa308-994f-40b1-8f42-f7af6f29672a service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:35.313099 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffa308-994f-40b1-8f42-f7af6f29672a service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:35.351514 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:35.376008 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306500b0-3c78-4135-a37f-4e52f13a99cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:35.376867 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306500b0-3c78-4135-a37f-4e52f13a99cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:35.377145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306500b0-3c78-4135-a37f-4e52f13a99cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:35.377413 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306500b0-3c78-4135-a37f-4e52f13a99cd admin admin] PATH_INFO: `/v3/projects/d0c5ea7bbd294bd7b7729aa26b6c715c` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:35.378570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306500b0-3c78-4135-a37f-4e52f13a99cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c5ea7bbd294bd7b7729aa26b6c715c)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:35.381015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306500b0-3c78-4135-a37f-4e52f13a99cd admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:35.501165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-306500b0-3c78-4135-a37f-4e52f13a99cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c5ea7bbd294bd7b7729aa26b6c715c', 'request_id': 'req-306500b0-3c78-4135-a37f-4e52f13a99cd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:35.601229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-306500b0-3c78-4135-a37f-4e52f13a99cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c5ea7bbd294bd7b7729aa26b6c715c', 'request_id': 'req-306500b0-3c78-4135-a37f-4e52f13a99cd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:35.607875 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-306500b0-3c78-4135-a37f-4e52f13a99cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c5ea7bbd294bd7b7729aa26b6c715c', 'request_id': 'req-306500b0-3c78-4135-a37f-4e52f13a99cd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:35.619316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-306500b0-3c78-4135-a37f-4e52f13a99cd admin admin] The token cache is being invalidate because project d0c5ea7bbd294bd7b7729aa26b6c715c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:35.619608 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-306500b0-3c78-4135-a37f-4e52f13a99cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:35.620259 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-306500b0-3c78-4135-a37f-4e52f13a99cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:35.621813 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1385/2765] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:18:35 2022] DELETE /identity/v3/projects/d0c5ea7bbd294bd7b7729aa26b6c715c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:35.627080 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077a2c9-d130-49a0-98d7-64f91fea3339 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:35.627558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077a2c9-d130-49a0-98d7-64f91fea3339 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:35.627739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077a2c9-d130-49a0-98d7-64f91fea3339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:35.647173 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9077a2c9-d130-49a0-98d7-64f91fea3339 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:35.648398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9077a2c9-d130-49a0-98d7-64f91fea3339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:35.709033 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1381/2766] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:35.716586 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598594b-f888-4fe0-8724-8686b9f9a95e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:35.716878 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598594b-f888-4fe0-8724-8686b9f9a95e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:35.717185 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598594b-f888-4fe0-8724-8686b9f9a95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:35.740251 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-4598594b-f888-4fe0-8724-8686b9f9a95e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:35.741488 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-4598594b-f888-4fe0-8724-8686b9f9a95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:35.754850 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1386/2767] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:35.757400 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077a2c9-d130-49a0-98d7-64f91fea3339 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:35.768123 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077a2c9-d130-49a0-98d7-64f91fea3339 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:35.773314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9606893-e450-4dcf-a40d-539d1c102231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:35.774128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9606893-e450-4dcf-a40d-539d1c102231 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:35.774339 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9606893-e450-4dcf-a40d-539d1c102231 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:35.774593 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9606893-e450-4dcf-a40d-539d1c102231 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:35.775396 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9606893-e450-4dcf-a40d-539d1c102231 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:35.775704 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9606893-e450-4dcf-a40d-539d1c102231 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:35.778384 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9606893-e450-4dcf-a40d-539d1c102231 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:35.781733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:35.798433 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1382/2768] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:18:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:35.803325 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598594b-f888-4fe0-8724-8686b9f9a95e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:35.814379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598594b-f888-4fe0-8724-8686b9f9a95e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:35.820015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118050c-3ae6-45fa-a2c2-27d271902999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:35.820886 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118050c-3ae6-45fa-a2c2-27d271902999 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:35.821239 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118050c-3ae6-45fa-a2c2-27d271902999 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:35.821579 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118050c-3ae6-45fa-a2c2-27d271902999 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:35.822660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118050c-3ae6-45fa-a2c2-27d271902999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:35.824971 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118050c-3ae6-45fa-a2c2-27d271902999 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:35.959064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1387/2769] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:18:35 2022] POST /identity/v3/projects => generated 372 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:35.963322 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9606893-e450-4dcf-a40d-539d1c102231 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:35.974321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9606893-e450-4dcf-a40d-539d1c102231 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:36.031772 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c545e02-07b9-49af-bf07-f782687625e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:36.032407 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c545e02-07b9-49af-bf07-f782687625e4 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:36.032774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c545e02-07b9-49af-bf07-f782687625e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:36.033041 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c545e02-07b9-49af-bf07-f782687625e4 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:36.033785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c545e02-07b9-49af-bf07-f782687625e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:36.036486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c545e02-07b9-49af-bf07-f782687625e4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:36.174664 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1383/2770] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:18:35 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:36.179827 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118050c-3ae6-45fa-a2c2-27d271902999 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:36.190345 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118050c-3ae6-45fa-a2c2-27d271902999 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:36.195887 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad3a15-032d-4173-a305-08577523445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:36.196569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad3a15-032d-4173-a305-08577523445e admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:36.196849 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad3a15-032d-4173-a305-08577523445e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:36.197121 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad3a15-032d-4173-a305-08577523445e admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:36.197844 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad3a15-032d-4173-a305-08577523445e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:36.199465 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad3a15-032d-4173-a305-08577523445e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:36.210890 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1388/2771] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:36 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:36.219334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c545e02-07b9-49af-bf07-f782687625e4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:36.230484 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c545e02-07b9-49af-bf07-f782687625e4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:36.235761 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292aad30-20b1-4c52-9707-85114ea94bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:36.236069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292aad30-20b1-4c52-9707-85114ea94bcd admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:36.236351 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292aad30-20b1-4c52-9707-85114ea94bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:36.236675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292aad30-20b1-4c52-9707-85114ea94bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:36.237414 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292aad30-20b1-4c52-9707-85114ea94bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:36.239293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292aad30-20b1-4c52-9707-85114ea94bcd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:36.257031 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1384/2772] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:36 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:18:36.261478 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad3a15-032d-4173-a305-08577523445e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:36.272243 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad3a15-032d-4173-a305-08577523445e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:36.277052 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec7ad2-c58c-4736-be00-a482c59a1eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:36.277939 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec7ad2-c58c-4736-be00-a482c59a1eb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:36.278217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec7ad2-c58c-4736-be00-a482c59a1eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:36.278485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec7ad2-c58c-4736-be00-a482c59a1eb0 admin admin] PATH_INFO: `/v3/projects/885d9016a02e4694aed3568d0c76d88f/users/0ef796c0658a4cd6b0f42dc32d8ca838/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:36.295711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec7ad2-c58c-4736-be00-a482c59a1eb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=885d9016a02e4694aed3568d0c76d88f, user_id=0ef796c0658a4cd6b0f42dc32d8ca838, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:36.298476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec7ad2-c58c-4736-be00-a482c59a1eb0 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:36.369455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1389/2773] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:18:36 2022] PUT /identity/v3/projects/885d9016a02e4694aed3568d0c76d88f/users/0ef796c0658a4cd6b0f42dc32d8ca838/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:36.379915 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e934747-8abe-4ffb-8929-4ee92af6a692 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:36.380660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e934747-8abe-4ffb-8929-4ee92af6a692 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:36.381092 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e934747-8abe-4ffb-8929-4ee92af6a692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:36.413149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2e934747-8abe-4ffb-8929-4ee92af6a692 None None] MFA Rules not processed for user `0ef796c0658a4cd6b0f42dc32d8ca838`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:36.414851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2e934747-8abe-4ffb-8929-4ee92af6a692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:36.491700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1385/2774] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:36.503205 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec7ad2-c58c-4736-be00-a482c59a1eb0 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:36.504985 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-13ec7ad2-c58c-4736-be00-a482c59a1eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:36.565818 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:36.589735 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec7ad2-c58c-4736-be00-a482c59a1eb0 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:36.596070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ae44c-f80e-4a19-ac90-e60f0f4f1ccf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:36.596808 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ae44c-f80e-4a19-ac90-e60f0f4f1ccf service nova] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:36.597071 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ae44c-f80e-4a19-ac90-e60f0f4f1ccf service nova] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:36.597346 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ae44c-f80e-4a19-ac90-e60f0f4f1ccf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:36.607956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ae44c-f80e-4a19-ac90-e60f0f4f1ccf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:36.609538 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ae44c-f80e-4a19-ac90-e60f0f4f1ccf service nova] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:36.630495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1390/2775] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:38.716674 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e934747-8abe-4ffb-8929-4ee92af6a692 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:38.718686 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2e934747-8abe-4ffb-8929-4ee92af6a692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:38.788162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e934747-8abe-4ffb-8929-4ee92af6a692 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:38.793639 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:38.794517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:38.794792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:38.795096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] PATH_INFO: `/v3/users/cdc1042ea3b54768ab87e1f66e74ec1c` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:38.797299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc1042ea3b54768ab87e1f66e74ec1c)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:38.799232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:38.802537 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:39.060977 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc1042ea3b54768ab87e1f66e74ec1c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.068052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc1042ea3b54768ab87e1f66e74ec1c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.068717 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc1042ea3b54768ab87e1f66e74ec1c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.080757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc1042ea3b54768ab87e1f66e74ec1c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.178057 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc1042ea3b54768ab87e1f66e74ec1c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.184783 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1386/2776] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:18:38 2022] DELETE /identity/v3/users/cdc1042ea3b54768ab87e1f66e74ec1c => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:39.450115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ae44c-f80e-4a19-ac90-e60f0f4f1ccf service nova] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:39.463978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ae44c-f80e-4a19-ac90-e60f0f4f1ccf service nova] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:39.523738 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:39.524661 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:39.524957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:39.525236 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] PATH_INFO: `/v3/projects/a50fbd483e2e4d8886ae5d6341af7dcb` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:39.526410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a50fbd483e2e4d8886ae5d6341af7dcb)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:39.528476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:39.676674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a50fbd483e2e4d8886ae5d6341af7dcb', 'request_id': 'req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.751626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a50fbd483e2e4d8886ae5d6341af7dcb', 'request_id': 'req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.759137 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a50fbd483e2e4d8886ae5d6341af7dcb', 'request_id': 'req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.769716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] The token cache is being invalidate because project a50fbd483e2e4d8886ae5d6341af7dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:39.770004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.771319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:39.773024 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1391/2777] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:18:39 2022] DELETE /identity/v3/projects/a50fbd483e2e4d8886ae5d6341af7dcb => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:39.777174 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:39.778645 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:39.792053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04240904-4eb3-48ed-92cf-c98cc6d4ccf8 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:39.852441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582f1f4-f375-49c9-a5c1-d6dbf1741cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:39.853348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582f1f4-f375-49c9-a5c1-d6dbf1741cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:39.853630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582f1f4-f375-49c9-a5c1-d6dbf1741cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:39.853905 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582f1f4-f375-49c9-a5c1-d6dbf1741cd7 admin admin] PATH_INFO: `/v3/users/233d5c99300642959bfa85a9e44f232e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:39.855172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582f1f4-f375-49c9-a5c1-d6dbf1741cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=233d5c99300642959bfa85a9e44f232e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:39.857084 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582f1f4-f375-49c9-a5c1-d6dbf1741cd7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:40.025700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:40.025700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:40.038036 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd21d96-1016-4343-bb8d-0a3e6ee12dab admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:40.042625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:40.045869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:40.045869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:40.045869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] PATH_INFO: `/v3/users/0ef796c0658a4cd6b0f42dc32d8ca838` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:40.045869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ef796c0658a4cd6b0f42dc32d8ca838)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:40.047342 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:40.084009 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2582f1f4-f375-49c9-a5c1-d6dbf1741cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '233d5c99300642959bfa85a9e44f232e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.090857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2582f1f4-f375-49c9-a5c1-d6dbf1741cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '233d5c99300642959bfa85a9e44f232e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.091586 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2582f1f4-f375-49c9-a5c1-d6dbf1741cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '233d5c99300642959bfa85a9e44f232e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.104175 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2582f1f4-f375-49c9-a5c1-d6dbf1741cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '233d5c99300642959bfa85a9e44f232e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.179474 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2582f1f4-f375-49c9-a5c1-d6dbf1741cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '233d5c99300642959bfa85a9e44f232e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.181923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:40.186770 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1387/2778] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:18:39 2022] DELETE /identity/v3/users/233d5c99300642959bfa85a9e44f232e => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:40.261956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ef796c0658a4cd6b0f42dc32d8ca838'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.269272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ef796c0658a4cd6b0f42dc32d8ca838'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.271542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ef796c0658a4cd6b0f42dc32d8ca838'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.275435 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:40.282706 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ef796c0658a4cd6b0f42dc32d8ca838'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.351254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ef796c0658a4cd6b0f42dc32d8ca838'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.357733 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1392/2779] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:18:40 2022] DELETE /identity/v3/users/0ef796c0658a4cd6b0f42dc32d8ca838 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:40.362492 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc5b6f-b453-44e8-82b3-e028e96eb8b8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:40.362782 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc5b6f-b453-44e8-82b3-e028e96eb8b8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:40.363095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc5b6f-b453-44e8-82b3-e028e96eb8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:40.383434 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-76fc5b6f-b453-44e8-82b3-e028e96eb8b8 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:40.384644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-76fc5b6f-b453-44e8-82b3-e028e96eb8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:40.434835 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:40.438233 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:40.448453 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1388/2780] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:40.458440 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc5b6f-b453-44e8-82b3-e028e96eb8b8 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:40.459860 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-76fc5b6f-b453-44e8-82b3-e028e96eb8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:40.521967 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e680d8-6263-4060-9ba3-9960e7c7cc25 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:40.522517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc5b6f-b453-44e8-82b3-e028e96eb8b8 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:40.526355 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8380f-5b22-4947-9767-44bf6c320dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:40.527044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8380f-5b22-4947-9767-44bf6c320dfb service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:40.527321 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8380f-5b22-4947-9767-44bf6c320dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:40.527524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:40.527644 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8380f-5b22-4947-9767-44bf6c320dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:40.528402 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:40.528684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:40.528978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61 admin admin] PATH_INFO: `/v3/projects/16d607c437d74fd4a328d6a33ef9fafe` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:40.530148 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16d607c437d74fd4a328d6a33ef9fafe)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:40.532160 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:40.538334 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8380f-5b22-4947-9767-44bf6c320dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:40.539792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8380f-5b22-4947-9767-44bf6c320dfb service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:40.549647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1389/2781] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:18:40.664407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16d607c437d74fd4a328d6a33ef9fafe', 'request_id': 'req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.768924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16d607c437d74fd4a328d6a33ef9fafe', 'request_id': 'req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.776229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16d607c437d74fd4a328d6a33ef9fafe', 'request_id': 'req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.787444 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61 admin admin] The token cache is being invalidate because project 16d607c437d74fd4a328d6a33ef9fafe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:40.787742 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.788802 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-2303e9ce-aae3-4ec7-aa74-c7e6cf7e9f61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:40.790261 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1393/2782] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:18:40 2022] DELETE /identity/v3/projects/16d607c437d74fd4a328d6a33ef9fafe => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:41.048096 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8380f-5b22-4947-9767-44bf6c320dfb service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.050677 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a7f8380f-5b22-4947-9767-44bf6c320dfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:41.064156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8380f-5b22-4947-9767-44bf6c320dfb service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.123924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.124797 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.125075 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.125347 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d admin admin] PATH_INFO: `/v3/projects/885d9016a02e4694aed3568d0c76d88f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.126622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=885d9016a02e4694aed3568d0c76d88f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.128527 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.245636 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:41.248441 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '885d9016a02e4694aed3568d0c76d88f', 'request_id': 'req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:41.356814 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '885d9016a02e4694aed3568d0c76d88f', 'request_id': 'req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:41.363438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '885d9016a02e4694aed3568d0c76d88f', 'request_id': 'req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:41.374643 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d admin admin] The token cache is being invalidate because project 885d9016a02e4694aed3568d0c76d88f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:41.374965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:41.375592 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-629e57df-65e8-44e7-bfb2-23af6b9f1c0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:41.377121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1390/2783] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:18:41 2022] DELETE /identity/v3/projects/885d9016a02e4694aed3568d0c76d88f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:41.382927 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5297d3-0c0c-4af7-b369-e5c31abe365e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.383220 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5297d3-0c0c-4af7-b369-e5c31abe365e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.383526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5297d3-0c0c-4af7-b369-e5c31abe365e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.403621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-0c5297d3-0c0c-4af7-b369-e5c31abe365e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:41.404822 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0c5297d3-0c0c-4af7-b369-e5c31abe365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:41.463476 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1394/2784] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:41.472307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77aad4c-315b-4150-b424-5a3dd33a638e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.472588 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77aad4c-315b-4150-b424-5a3dd33a638e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.472857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77aad4c-315b-4150-b424-5a3dd33a638e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.491745 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a77aad4c-315b-4150-b424-5a3dd33a638e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:41.492943 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a77aad4c-315b-4150-b424-5a3dd33a638e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:41.500942 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1391/2785] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:41.505641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5297d3-0c0c-4af7-b369-e5c31abe365e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.515533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5297d3-0c0c-4af7-b369-e5c31abe365e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.519973 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872f11f-37f5-4f30-a867-d094f922e95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.521159 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872f11f-37f5-4f30-a867-d094f922e95d admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.521639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872f11f-37f5-4f30-a867-d094f922e95d admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.521876 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872f11f-37f5-4f30-a867-d094f922e95d admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.522524 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872f11f-37f5-4f30-a867-d094f922e95d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:41.523115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872f11f-37f5-4f30-a867-d094f922e95d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.524362 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872f11f-37f5-4f30-a867-d094f922e95d admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.531436 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:41.536917 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1395/2786] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:18:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:41.541649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77aad4c-315b-4150-b424-5a3dd33a638e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.551050 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77aad4c-315b-4150-b424-5a3dd33a638e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.557125 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c828eb-e214-4abb-81b8-25168df943a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.558031 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c828eb-e214-4abb-81b8-25168df943a9 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.562000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c828eb-e214-4abb-81b8-25168df943a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.562000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c828eb-e214-4abb-81b8-25168df943a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.562000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c828eb-e214-4abb-81b8-25168df943a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.562000 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c828eb-e214-4abb-81b8-25168df943a9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.644228 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1392/2787] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:18:41 2022] POST /identity/v3/projects => generated 388 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:41.648885 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872f11f-37f5-4f30-a867-d094f922e95d admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.658894 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872f11f-37f5-4f30-a867-d094f922e95d admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.716082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1376e1-7d37-440a-8bb7-a2229a13a371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.716757 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1376e1-7d37-440a-8bb7-a2229a13a371 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.717039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1376e1-7d37-440a-8bb7-a2229a13a371 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.717315 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1376e1-7d37-440a-8bb7-a2229a13a371 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.717996 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1376e1-7d37-440a-8bb7-a2229a13a371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.719936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1376e1-7d37-440a-8bb7-a2229a13a371 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.805371 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1396/2788] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:18:41 2022] POST /identity/v3/users => generated 295 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:41.809843 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c828eb-e214-4abb-81b8-25168df943a9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.821211 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c828eb-e214-4abb-81b8-25168df943a9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.825878 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84665cbf-52c9-446e-b3ec-e7f3c7eda650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.826536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84665cbf-52c9-446e-b3ec-e7f3c7eda650 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.826845 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84665cbf-52c9-446e-b3ec-e7f3c7eda650 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.827126 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84665cbf-52c9-446e-b3ec-e7f3c7eda650 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.827848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84665cbf-52c9-446e-b3ec-e7f3c7eda650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.829454 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84665cbf-52c9-446e-b3ec-e7f3c7eda650 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.842063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1393/2789] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:41 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:41.846029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1376e1-7d37-440a-8bb7-a2229a13a371 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.856013 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1376e1-7d37-440a-8bb7-a2229a13a371 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.860534 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05033f-89b8-45e3-a1c9-08111234e9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.861217 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05033f-89b8-45e3-a1c9-08111234e9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.861492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05033f-89b8-45e3-a1c9-08111234e9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.861761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05033f-89b8-45e3-a1c9-08111234e9d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.862471 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05033f-89b8-45e3-a1c9-08111234e9d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.864082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05033f-89b8-45e3-a1c9-08111234e9d2 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.875107 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1397/2790] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:41 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:41.879226 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84665cbf-52c9-446e-b3ec-e7f3c7eda650 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:41.889589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84665cbf-52c9-446e-b3ec-e7f3c7eda650 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:41.894099 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428c058-a629-49ff-b9ab-aed4637bd8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:41.895058 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428c058-a629-49ff-b9ab-aed4637bd8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.895281 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428c058-a629-49ff-b9ab-aed4637bd8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.895571 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428c058-a629-49ff-b9ab-aed4637bd8e9 admin admin] PATH_INFO: `/v3/projects/b3342882aa0d4c60a7f0812257611e40/users/c309aa30b49a4659b18b602aee64c1ec/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:41.909939 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428c058-a629-49ff-b9ab-aed4637bd8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3342882aa0d4c60a7f0812257611e40, user_id=c309aa30b49a4659b18b602aee64c1ec, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:41.912637 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428c058-a629-49ff-b9ab-aed4637bd8e9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:41.978063 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1394/2791] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:18:41 2022] PUT /identity/v3/projects/b3342882aa0d4c60a7f0812257611e40/users/c309aa30b49a4659b18b602aee64c1ec/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:41.988142 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f26f26-2946-492d-b490-d73676e7f58b None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:41.988424 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f26f26-2946-492d-b490-d73676e7f58b None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:41.988721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f26f26-2946-492d-b490-d73676e7f58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:42.015149 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-72f26f26-2946-492d-b490-d73676e7f58b None None] MFA Rules not processed for user `c309aa30b49a4659b18b602aee64c1ec`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:42.016358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-72f26f26-2946-492d-b490-d73676e7f58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:42.079260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1398/2792] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:42.083903 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428c058-a629-49ff-b9ab-aed4637bd8e9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:42.144806 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428c058-a629-49ff-b9ab-aed4637bd8e9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:42.149825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd0e12-bb62-4d89-aa0f-9a59db48e55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:42.150417 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd0e12-bb62-4d89-aa0f-9a59db48e55f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:42.150698 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd0e12-bb62-4d89-aa0f-9a59db48e55f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:42.151002 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd0e12-bb62-4d89-aa0f-9a59db48e55f admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:42.151687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd0e12-bb62-4d89-aa0f-9a59db48e55f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:42.153589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd0e12-bb62-4d89-aa0f-9a59db48e55f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:42.249800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1395/2793] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:18:42 2022] POST /identity/v3/projects => generated 386 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:42.253923 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f26f26-2946-492d-b490-d73676e7f58b None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:42.264934 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f26f26-2946-492d-b490-d73676e7f58b None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:42.321607 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7816ad-e639-4f42-9606-dda5bcdc9d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:42.322289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7816ad-e639-4f42-9606-dda5bcdc9d07 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:42.322566 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7816ad-e639-4f42-9606-dda5bcdc9d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:42.322862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7816ad-e639-4f42-9606-dda5bcdc9d07 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:42.323559 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7816ad-e639-4f42-9606-dda5bcdc9d07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:42.325451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7816ad-e639-4f42-9606-dda5bcdc9d07 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:42.336077 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:42.425670 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1399/2794] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:18:42 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:42.429795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd0e12-bb62-4d89-aa0f-9a59db48e55f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:42.440698 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd0e12-bb62-4d89-aa0f-9a59db48e55f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:42.445368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f69e0e-062d-4b46-b335-9f1ba170dda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:42.446033 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f69e0e-062d-4b46-b335-9f1ba170dda1 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:42.446309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f69e0e-062d-4b46-b335-9f1ba170dda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:42.446581 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f69e0e-062d-4b46-b335-9f1ba170dda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:42.447330 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f69e0e-062d-4b46-b335-9f1ba170dda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:42.448908 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f69e0e-062d-4b46-b335-9f1ba170dda1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:42.460139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1396/2795] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:42 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:42.464373 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7816ad-e639-4f42-9606-dda5bcdc9d07 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:42.474904 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7816ad-e639-4f42-9606-dda5bcdc9d07 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:42.479508 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea870c4-4788-4cb8-a8c7-5f53c95f4bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:42.480379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea870c4-4788-4cb8-a8c7-5f53c95f4bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:42.480658 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea870c4-4788-4cb8-a8c7-5f53c95f4bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:42.480977 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea870c4-4788-4cb8-a8c7-5f53c95f4bd2 admin admin] PATH_INFO: `/v3/projects/d5b1a88d50f04e06aa06186d8eb8b36a/users/6c184458582c484ba3a62cf9578210ae/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:42.496181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea870c4-4788-4cb8-a8c7-5f53c95f4bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b1a88d50f04e06aa06186d8eb8b36a, user_id=6c184458582c484ba3a62cf9578210ae, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:42.498774 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea870c4-4788-4cb8-a8c7-5f53c95f4bd2 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:42.563038 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1400/2796] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:18:42 2022] PUT /identity/v3/projects/d5b1a88d50f04e06aa06186d8eb8b36a/users/6c184458582c484ba3a62cf9578210ae/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:42.567257 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f69e0e-062d-4b46-b335-9f1ba170dda1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:42.578271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f69e0e-062d-4b46-b335-9f1ba170dda1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:42.619711 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:42.634143 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b0532-a80e-41c2-83c4-b8ad115d7ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:42.634809 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b0532-a80e-41c2-83c4-b8ad115d7ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:42.635118 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b0532-a80e-41c2-83c4-b8ad115d7ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:42.635394 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b0532-a80e-41c2-83c4-b8ad115d7ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:42.636106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b0532-a80e-41c2-83c4-b8ad115d7ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:42.637692 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b0532-a80e-41c2-83c4-b8ad115d7ec4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:42.648083 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1397/2797] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:42 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:42.652215 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea870c4-4788-4cb8-a8c7-5f53c95f4bd2 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:42.666452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea870c4-4788-4cb8-a8c7-5f53c95f4bd2 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:42.667879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17fb52-d1d4-44b1-b25f-163d4bb2b751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:42.668778 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17fb52-d1d4-44b1-b25f-163d4bb2b751 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:42.669041 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17fb52-d1d4-44b1-b25f-163d4bb2b751 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:42.669312 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17fb52-d1d4-44b1-b25f-163d4bb2b751 admin admin] PATH_INFO: `/v3/projects/d5b1a88d50f04e06aa06186d8eb8b36a/users/6c184458582c484ba3a62cf9578210ae/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:42.671492 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17fb52-d1d4-44b1-b25f-163d4bb2b751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b1a88d50f04e06aa06186d8eb8b36a, user_id=6c184458582c484ba3a62cf9578210ae, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:42.674207 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17fb52-d1d4-44b1-b25f-163d4bb2b751 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:42.685198 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1401/2798] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:18:42 2022] PUT /identity/v3/projects/d5b1a88d50f04e06aa06186d8eb8b36a/users/6c184458582c484ba3a62cf9578210ae/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:42.695957 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5abbc2-4022-4b4d-bc4a-cc0dfc9572c1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:42.696270 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5abbc2-4022-4b4d-bc4a-cc0dfc9572c1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:42.696549 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5abbc2-4022-4b4d-bc4a-cc0dfc9572c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:42.723439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-fa5abbc2-4022-4b4d-bc4a-cc0dfc9572c1 None None] MFA Rules not processed for user `6c184458582c484ba3a62cf9578210ae`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:42.725013 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fa5abbc2-4022-4b4d-bc4a-cc0dfc9572c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:42.795242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1398/2799] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:42.806560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17fb52-d1d4-44b1-b25f-163d4bb2b751 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:42.808583 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1c17fb52-d1d4-44b1-b25f-163d4bb2b751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:42.867059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17fb52-d1d4-44b1-b25f-163d4bb2b751 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:42.870659 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c1724-200e-41c3-91ad-d8a5f607d137 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:42.871330 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c1724-200e-41c3-91ad-d8a5f607d137 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:42.871610 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c1724-200e-41c3-91ad-d8a5f607d137 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:42.871880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c1724-200e-41c3-91ad-d8a5f607d137 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:42.881546 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c1724-200e-41c3-91ad-d8a5f607d137 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:42.882991 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c1724-200e-41c3-91ad-d8a5f607d137 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:42.892311 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1402/2800] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:42.956236 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5abbc2-4022-4b4d-bc4a-cc0dfc9572c1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:43.018288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5abbc2-4022-4b4d-bc4a-cc0dfc9572c1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:43.023720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:43.024615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:43.024893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:43.025197 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] PATH_INFO: `/v3/users/c309aa30b49a4659b18b602aee64c1ec` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:43.026395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c309aa30b49a4659b18b602aee64c1ec)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:43.028349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:43.231052 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c309aa30b49a4659b18b602aee64c1ec'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:43.237813 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c309aa30b49a4659b18b602aee64c1ec'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:43.238478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c309aa30b49a4659b18b602aee64c1ec'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:43.246647 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:43.256308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c309aa30b49a4659b18b602aee64c1ec'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:43.360705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c309aa30b49a4659b18b602aee64c1ec'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:43.368213 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1399/2801] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:18:42 2022] DELETE /identity/v3/users/c309aa30b49a4659b18b602aee64c1ec => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:43.372880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87559d5-ccdc-4d2e-a097-96538572d5ee None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:43.373176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87559d5-ccdc-4d2e-a097-96538572d5ee None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:43.373450 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87559d5-ccdc-4d2e-a097-96538572d5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:43.391139 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-b87559d5-ccdc-4d2e-a097-96538572d5ee None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:43.392334 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b87559d5-ccdc-4d2e-a097-96538572d5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:43.450292 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1403/2802] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:43.460399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:43.461786 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:43.520991 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f426b06-2220-4e7f-82fc-fe39629ba197 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:43.524760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e210b-bbdf-40de-852a-1e831715572d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:43.525420 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e210b-bbdf-40de-852a-1e831715572d service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:43.525851 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e210b-bbdf-40de-852a-1e831715572d service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:43.526131 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e210b-bbdf-40de-852a-1e831715572d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:43.535350 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e210b-bbdf-40de-852a-1e831715572d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:43.537757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e210b-bbdf-40de-852a-1e831715572d service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:43.548132 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1400/2803] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:43.962581 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87559d5-ccdc-4d2e-a097-96538572d5ee None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:43.964115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-b87559d5-ccdc-4d2e-a097-96538572d5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:43.978694 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87559d5-ccdc-4d2e-a097-96538572d5ee None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:43.983225 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:43.984626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:43.985066 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:43.985344 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] PATH_INFO: `/v3/projects/b3342882aa0d4c60a7f0812257611e40` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:43.986645 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3342882aa0d4c60a7f0812257611e40)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:43.988529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:44.134777 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3342882aa0d4c60a7f0812257611e40', 'request_id': 'req-068514ed-0271-46fd-a720-f1625182a993'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.208761 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3342882aa0d4c60a7f0812257611e40', 'request_id': 'req-068514ed-0271-46fd-a720-f1625182a993'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.210152 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:44.215958 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3342882aa0d4c60a7f0812257611e40', 'request_id': 'req-068514ed-0271-46fd-a720-f1625182a993'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.226007 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] The token cache is being invalidate because project b3342882aa0d4c60a7f0812257611e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:44.226291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.226956 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.228465 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1404/2804] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:18:43 2022] DELETE /identity/v3/projects/b3342882aa0d4c60a7f0812257611e40 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:44.232538 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e210b-bbdf-40de-852a-1e831715572d service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:44.242369 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e210b-bbdf-40de-852a-1e831715572d service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:44.297127 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea80cde-1616-4a77-8922-3df7502897ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:44.297984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea80cde-1616-4a77-8922-3df7502897ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:44.298263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea80cde-1616-4a77-8922-3df7502897ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:44.298569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea80cde-1616-4a77-8922-3df7502897ff admin admin] PATH_INFO: `/v3/users/6c184458582c484ba3a62cf9578210ae` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:44.299755 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea80cde-1616-4a77-8922-3df7502897ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c184458582c484ba3a62cf9578210ae)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:44.301665 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea80cde-1616-4a77-8922-3df7502897ff admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:44.460946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0ea80cde-1616-4a77-8922-3df7502897ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c184458582c484ba3a62cf9578210ae'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.468053 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0ea80cde-1616-4a77-8922-3df7502897ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c184458582c484ba3a62cf9578210ae'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.468803 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0ea80cde-1616-4a77-8922-3df7502897ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c184458582c484ba3a62cf9578210ae'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.480953 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0ea80cde-1616-4a77-8922-3df7502897ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c184458582c484ba3a62cf9578210ae'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.561949 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0ea80cde-1616-4a77-8922-3df7502897ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c184458582c484ba3a62cf9578210ae'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:44.569104 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1401/2805] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:18:44 2022] DELETE /identity/v3/users/6c184458582c484ba3a62cf9578210ae => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:44.968114 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:44.969786 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:44.984109 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068514ed-0271-46fd-a720-f1625182a993 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.049467 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efacf394-1095-4bc7-95aa-c5a510948469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.050322 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efacf394-1095-4bc7-95aa-c5a510948469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.050637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efacf394-1095-4bc7-95aa-c5a510948469 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.050940 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efacf394-1095-4bc7-95aa-c5a510948469 admin admin] PATH_INFO: `/v3/projects/d5b1a88d50f04e06aa06186d8eb8b36a` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.052687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efacf394-1095-4bc7-95aa-c5a510948469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5b1a88d50f04e06aa06186d8eb8b36a)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.054549 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efacf394-1095-4bc7-95aa-c5a510948469 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.154463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-efacf394-1095-4bc7-95aa-c5a510948469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5b1a88d50f04e06aa06186d8eb8b36a', 'request_id': 'req-efacf394-1095-4bc7-95aa-c5a510948469'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.228714 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-efacf394-1095-4bc7-95aa-c5a510948469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5b1a88d50f04e06aa06186d8eb8b36a', 'request_id': 'req-efacf394-1095-4bc7-95aa-c5a510948469'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.235343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-efacf394-1095-4bc7-95aa-c5a510948469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5b1a88d50f04e06aa06186d8eb8b36a', 'request_id': 'req-efacf394-1095-4bc7-95aa-c5a510948469'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.246439 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-efacf394-1095-4bc7-95aa-c5a510948469 admin admin] The token cache is being invalidate because project d5b1a88d50f04e06aa06186d8eb8b36a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:45.246665 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-efacf394-1095-4bc7-95aa-c5a510948469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.247320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-efacf394-1095-4bc7-95aa-c5a510948469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:45.248869 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1405/2806] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:18:44 2022] DELETE /identity/v3/projects/d5b1a88d50f04e06aa06186d8eb8b36a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:45.253823 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4b1bc-7c14-4f90-8c49-94d9fcf8702f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.254318 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4b1bc-7c14-4f90-8c49-94d9fcf8702f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.254420 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4b1bc-7c14-4f90-8c49-94d9fcf8702f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.266707 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:45.274333 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1fc4b1bc-7c14-4f90-8c49-94d9fcf8702f None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:45.275573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1fc4b1bc-7c14-4f90-8c49-94d9fcf8702f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:45.335700 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1402/2807] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:45.344913 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01b650-2915-4cd0-8daa-008749cb6ad3 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.345328 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01b650-2915-4cd0-8daa-008749cb6ad3 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.345716 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01b650-2915-4cd0-8daa-008749cb6ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.365287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-1f01b650-2915-4cd0-8daa-008749cb6ad3 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:45.366469 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1f01b650-2915-4cd0-8daa-008749cb6ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:45.374339 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1406/2808] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:45.378582 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4b1bc-7c14-4f90-8c49-94d9fcf8702f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.388329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4b1bc-7c14-4f90-8c49-94d9fcf8702f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.393303 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d7475-4c2c-447d-a10f-844d4e5a32e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.394010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d7475-4c2c-447d-a10f-844d4e5a32e9 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.394289 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d7475-4c2c-447d-a10f-844d4e5a32e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.394558 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d7475-4c2c-447d-a10f-844d4e5a32e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.395308 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d7475-4c2c-447d-a10f-844d4e5a32e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:45.395611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d7475-4c2c-447d-a10f-844d4e5a32e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.397951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d7475-4c2c-447d-a10f-844d4e5a32e9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.409427 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1403/2809] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:18:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:18:45.414338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01b650-2915-4cd0-8daa-008749cb6ad3 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.423370 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01b650-2915-4cd0-8daa-008749cb6ad3 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.427906 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8931da-f8a2-4772-a6c6-f5e0ef034759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.428648 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8931da-f8a2-4772-a6c6-f5e0ef034759 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.428957 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8931da-f8a2-4772-a6c6-f5e0ef034759 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.429235 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8931da-f8a2-4772-a6c6-f5e0ef034759 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.429916 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8931da-f8a2-4772-a6c6-f5e0ef034759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.431858 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8931da-f8a2-4772-a6c6-f5e0ef034759 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.516764 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1407/2810] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:18:45 2022] POST /identity/v3/projects => generated 368 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:45.521010 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d7475-4c2c-447d-a10f-844d4e5a32e9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.531703 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d7475-4c2c-447d-a10f-844d4e5a32e9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.588606 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f55c7-5444-4793-ad30-79eff70d873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.589274 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f55c7-5444-4793-ad30-79eff70d873a admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.589552 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f55c7-5444-4793-ad30-79eff70d873a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.589824 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f55c7-5444-4793-ad30-79eff70d873a admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.590509 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f55c7-5444-4793-ad30-79eff70d873a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.592463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f55c7-5444-4793-ad30-79eff70d873a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.663925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1404/2811] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:18:45 2022] POST /identity/v3/users => generated 285 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:45.668030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8931da-f8a2-4772-a6c6-f5e0ef034759 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.678351 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8931da-f8a2-4772-a6c6-f5e0ef034759 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.683230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47701d-c8ad-46d8-8ae9-e868c1dce6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.683889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47701d-c8ad-46d8-8ae9-e868c1dce6aa admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.684171 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47701d-c8ad-46d8-8ae9-e868c1dce6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.684446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47701d-c8ad-46d8-8ae9-e868c1dce6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.685167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47701d-c8ad-46d8-8ae9-e868c1dce6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.686783 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47701d-c8ad-46d8-8ae9-e868c1dce6aa admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.697848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1408/2812] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:45 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:45.701879 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f55c7-5444-4793-ad30-79eff70d873a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.712666 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f55c7-5444-4793-ad30-79eff70d873a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.717535 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da6a1a-8e21-47a3-976b-2b4caa1c95a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.718192 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da6a1a-8e21-47a3-976b-2b4caa1c95a8 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.718472 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da6a1a-8e21-47a3-976b-2b4caa1c95a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.718744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da6a1a-8e21-47a3-976b-2b4caa1c95a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.719494 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da6a1a-8e21-47a3-976b-2b4caa1c95a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.721082 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da6a1a-8e21-47a3-976b-2b4caa1c95a8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.731255 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1405/2813] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:45 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:45.735411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47701d-c8ad-46d8-8ae9-e868c1dce6aa admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.745331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47701d-c8ad-46d8-8ae9-e868c1dce6aa admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:45.750051 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd675b1-1c06-44ca-b77e-5db1ec0834bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:45.750969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd675b1-1c06-44ca-b77e-5db1ec0834bd admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.751653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd675b1-1c06-44ca-b77e-5db1ec0834bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.751653 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd675b1-1c06-44ca-b77e-5db1ec0834bd admin admin] PATH_INFO: `/v3/projects/146d0d7d330b444eab7230789f9a8b39/users/dfa459d0ee204f1c91e120cb28498a0e/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.766252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd675b1-1c06-44ca-b77e-5db1ec0834bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=146d0d7d330b444eab7230789f9a8b39, user_id=dfa459d0ee204f1c91e120cb28498a0e, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:45.768962 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd675b1-1c06-44ca-b77e-5db1ec0834bd admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:45.837785 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1409/2814] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:18:45 2022] PUT /identity/v3/projects/146d0d7d330b444eab7230789f9a8b39/users/dfa459d0ee204f1c91e120cb28498a0e/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:18:45.851562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5be6c-8ad4-4516-807a-e845eca47bbb None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:45.851854 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5be6c-8ad4-4516-807a-e845eca47bbb None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:45.852136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5be6c-8ad4-4516-807a-e845eca47bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:45.878767 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-76a5be6c-8ad4-4516-807a-e845eca47bbb None None] MFA Rules not processed for user `dfa459d0ee204f1c91e120cb28498a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:45.880020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-76a5be6c-8ad4-4516-807a-e845eca47bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:45.948139 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1406/2815] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:45.958572 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd675b1-1c06-44ca-b77e-5db1ec0834bd admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:45.961207 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-afd675b1-1c06-44ca-b77e-5db1ec0834bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:45.975127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:46.019662 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd675b1-1c06-44ca-b77e-5db1ec0834bd admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:46.023695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8a411-e7d1-47ed-a957-4f7df138f4e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:46.024338 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8a411-e7d1-47ed-a957-4f7df138f4e7 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:46.024616 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8a411-e7d1-47ed-a957-4f7df138f4e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:46.024884 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8a411-e7d1-47ed-a957-4f7df138f4e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:46.033013 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a8a411-e7d1-47ed-a957-4f7df138f4e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:46.034484 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a8a411-e7d1-47ed-a957-4f7df138f4e7 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:46.044243 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1410/2816] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:49.991447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5be6c-8ad4-4516-807a-e845eca47bbb None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:49.995664 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:50.053889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5be6c-8ad4-4516-807a-e845eca47bbb None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:50.058857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:50.059743 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:50.060159 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:50.060849 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] PATH_INFO: `/v3/users/dfa459d0ee204f1c91e120cb28498a0e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:50.062033 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa459d0ee204f1c91e120cb28498a0e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:50.063986 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:50.252948 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa459d0ee204f1c91e120cb28498a0e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:50.260741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa459d0ee204f1c91e120cb28498a0e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:50.261456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa459d0ee204f1c91e120cb28498a0e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:50.274095 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa459d0ee204f1c91e120cb28498a0e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:50.349170 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa459d0ee204f1c91e120cb28498a0e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:50.355882 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1407/2817] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:18:49 2022] DELETE /identity/v3/users/dfa459d0ee204f1c91e120cb28498a0e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:50.360407 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f25c1d-db66-40a8-a757-bbfab154ab3a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:50.360915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f25c1d-db66-40a8-a757-bbfab154ab3a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:50.361031 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f25c1d-db66-40a8-a757-bbfab154ab3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:50.378848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a2f25c1d-db66-40a8-a757-bbfab154ab3a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:50.380067 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a2f25c1d-db66-40a8-a757-bbfab154ab3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:50.442748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1411/2818] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:18:50.453819 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:50.455136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:50.523391 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d75ae-2092-47cf-98eb-5b6bcc53a321 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:50.527946 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2fd19-94fc-4f41-bc90-8b2e0de1221e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:50.528596 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2fd19-94fc-4f41-bc90-8b2e0de1221e service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:50.528877 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2fd19-94fc-4f41-bc90-8b2e0de1221e service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:50.529151 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2fd19-94fc-4f41-bc90-8b2e0de1221e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:50.539412 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2fd19-94fc-4f41-bc90-8b2e0de1221e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:50.540872 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2fd19-94fc-4f41-bc90-8b2e0de1221e service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:50.551396 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1408/2819] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:50.969145 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f25c1d-db66-40a8-a757-bbfab154ab3a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:50.970567 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a2f25c1d-db66-40a8-a757-bbfab154ab3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:50.983230 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f25c1d-db66-40a8-a757-bbfab154ab3a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:50.988266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e86d3b-4724-4b07-9433-de45a1dcdb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:50.989141 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86d3b-4724-4b07-9433-de45a1dcdb32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:50.989423 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86d3b-4724-4b07-9433-de45a1dcdb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:50.989695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86d3b-4724-4b07-9433-de45a1dcdb32 admin admin] PATH_INFO: `/v3/projects/146d0d7d330b444eab7230789f9a8b39` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:50.991574 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e86d3b-4724-4b07-9433-de45a1dcdb32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=146d0d7d330b444eab7230789f9a8b39)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:50.993873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e86d3b-4724-4b07-9433-de45a1dcdb32 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:51.127004 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50e86d3b-4724-4b07-9433-de45a1dcdb32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '146d0d7d330b444eab7230789f9a8b39', 'request_id': 'req-50e86d3b-4724-4b07-9433-de45a1dcdb32'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:51.183359 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50e86d3b-4724-4b07-9433-de45a1dcdb32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '146d0d7d330b444eab7230789f9a8b39', 'request_id': 'req-50e86d3b-4724-4b07-9433-de45a1dcdb32'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:51.189815 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50e86d3b-4724-4b07-9433-de45a1dcdb32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '146d0d7d330b444eab7230789f9a8b39', 'request_id': 'req-50e86d3b-4724-4b07-9433-de45a1dcdb32'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:51.200045 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50e86d3b-4724-4b07-9433-de45a1dcdb32 admin admin] The token cache is being invalidate because project 146d0d7d330b444eab7230789f9a8b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92082) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:18:51.200325 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50e86d3b-4724-4b07-9433-de45a1dcdb32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:51.200953 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-50e86d3b-4724-4b07-9433-de45a1dcdb32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:18:51.202477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1412/2820] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:18:50 2022] DELETE /identity/v3/projects/146d0d7d330b444eab7230789f9a8b39 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:51.207646 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dec394-f888-4bf0-8fd4-1eff88393afc None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:51.208106 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dec394-f888-4bf0-8fd4-1eff88393afc None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:51.208231 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dec394-f888-4bf0-8fd4-1eff88393afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:51.226595 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-58dec394-f888-4bf0-8fd4-1eff88393afc None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:51.227832 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-58dec394-f888-4bf0-8fd4-1eff88393afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:51.248034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:51.288981 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1409/2821] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:51.296963 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bba992-00d0-4bf2-a9b3-1d16c8b7d17a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:51.297287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bba992-00d0-4bf2-a9b3-1d16c8b7d17a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:51.297560 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bba992-00d0-4bf2-a9b3-1d16c8b7d17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:51.315884 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-32bba992-00d0-4bf2-a9b3-1d16c8b7d17a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:51.317069 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-32bba992-00d0-4bf2-a9b3-1d16c8b7d17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:51.325189 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1413/2822] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:51.329389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dec394-f888-4bf0-8fd4-1eff88393afc None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:51.340273 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dec394-f888-4bf0-8fd4-1eff88393afc None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:51.344951 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8c798-386d-4981-b4ae-a6e172220319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:51.347605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8c798-386d-4981-b4ae-a6e172220319 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:51.347605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8c798-386d-4981-b4ae-a6e172220319 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:51.347605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8c798-386d-4981-b4ae-a6e172220319 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:51.347605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8c798-386d-4981-b4ae-a6e172220319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:18:51.347605 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8c798-386d-4981-b4ae-a6e172220319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:51.348729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8c798-386d-4981-b4ae-a6e172220319 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:51.360934 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1410/2823] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:18:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:18:51.365213 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bba992-00d0-4bf2-a9b3-1d16c8b7d17a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:51.374485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bba992-00d0-4bf2-a9b3-1d16c8b7d17a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:51.379085 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307cb5f-b92d-48f4-b50e-abaf6aa97ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:51.379732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307cb5f-b92d-48f4-b50e-abaf6aa97ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:51.380006 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307cb5f-b92d-48f4-b50e-abaf6aa97ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:51.380272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307cb5f-b92d-48f4-b50e-abaf6aa97ba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:51.381181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307cb5f-b92d-48f4-b50e-abaf6aa97ba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:51.383167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307cb5f-b92d-48f4-b50e-abaf6aa97ba4 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:51.467184 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1414/2824] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:18:51 2022] POST /identity/v3/projects => generated 378 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:51.471491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8c798-386d-4981-b4ae-a6e172220319 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:51.482426 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8c798-386d-4981-b4ae-a6e172220319 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:51.539071 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e8cc3-a5e0-4825-b8dc-5e0ad0473395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:51.539752 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e8cc3-a5e0-4825-b8dc-5e0ad0473395 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:51.540034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e8cc3-a5e0-4825-b8dc-5e0ad0473395 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:51.540307 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e8cc3-a5e0-4825-b8dc-5e0ad0473395 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:51.540994 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e8cc3-a5e0-4825-b8dc-5e0ad0473395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:51.543007 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e8cc3-a5e0-4825-b8dc-5e0ad0473395 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:51.619751 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1411/2825] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:18:51 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:51.625321 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307cb5f-b92d-48f4-b50e-abaf6aa97ba4 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:51.635744 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307cb5f-b92d-48f4-b50e-abaf6aa97ba4 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:51.640307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10e91c-9d9c-493a-a240-146fa71a3dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:51.640967 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10e91c-9d9c-493a-a240-146fa71a3dba admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:51.641243 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10e91c-9d9c-493a-a240-146fa71a3dba admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:51.641512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10e91c-9d9c-493a-a240-146fa71a3dba admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:51.642228 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10e91c-9d9c-493a-a240-146fa71a3dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:51.643866 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10e91c-9d9c-493a-a240-146fa71a3dba admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:51.645889 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:51.655039 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1415/2826] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:51 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:51.666024 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e8cc3-a5e0-4825-b8dc-5e0ad0473395 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:51.676059 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e8cc3-a5e0-4825-b8dc-5e0ad0473395 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:51.680737 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5c7ff-63c5-4a19-b8a8-2a076df2298b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:51.681406 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5c7ff-63c5-4a19-b8a8-2a076df2298b admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:51.681684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5c7ff-63c5-4a19-b8a8-2a076df2298b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:51.681955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5c7ff-63c5-4a19-b8a8-2a076df2298b admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:51.682955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5c7ff-63c5-4a19-b8a8-2a076df2298b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:51.684536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5c7ff-63c5-4a19-b8a8-2a076df2298b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:51.695348 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1412/2827] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:51 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:51.699446 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10e91c-9d9c-493a-a240-146fa71a3dba admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:51.710105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10e91c-9d9c-493a-a240-146fa71a3dba admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:51.715393 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730bb93-b782-4c55-ba30-671af5f8ad08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:51.716280 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730bb93-b782-4c55-ba30-671af5f8ad08 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:51.719448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730bb93-b782-4c55-ba30-671af5f8ad08 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:51.719448 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730bb93-b782-4c55-ba30-671af5f8ad08 admin admin] PATH_INFO: `/v3/projects/483c00eb7e46472493eb9ee4baa16090/users/c64dd84abd394dc5a93e774c15c6d141/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:51.731307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730bb93-b782-4c55-ba30-671af5f8ad08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=483c00eb7e46472493eb9ee4baa16090, user_id=c64dd84abd394dc5a93e774c15c6d141, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:51.733920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730bb93-b782-4c55-ba30-671af5f8ad08 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:51.796406 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1416/2828] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:18:51 2022] PUT /identity/v3/projects/483c00eb7e46472493eb9ee4baa16090/users/c64dd84abd394dc5a93e774c15c6d141/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:51.806524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e27f7-d327-4634-86e3-62cd8aabed6c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:51.806840 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e27f7-d327-4634-86e3-62cd8aabed6c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:51.807121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e27f7-d327-4634-86e3-62cd8aabed6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:51.837148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8d7e27f7-d327-4634-86e3-62cd8aabed6c None None] MFA Rules not processed for user `c64dd84abd394dc5a93e774c15c6d141`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:51.838340 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8d7e27f7-d327-4634-86e3-62cd8aabed6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:51.903847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1413/2829] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:51.908279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730bb93-b782-4c55-ba30-671af5f8ad08 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:51.970340 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730bb93-b782-4c55-ba30-671af5f8ad08 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:51.975042 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737e25f-f37e-4220-9f10-1b9414ec9724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:51.975700 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737e25f-f37e-4220-9f10-1b9414ec9724 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:51.975982 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737e25f-f37e-4220-9f10-1b9414ec9724 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:51.976254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737e25f-f37e-4220-9f10-1b9414ec9724 admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:51.976928 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737e25f-f37e-4220-9f10-1b9414ec9724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:51.978801 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737e25f-f37e-4220-9f10-1b9414ec9724 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.107009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1417/2830] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:18:51 2022] POST /identity/v3/projects => generated 378 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:52.111327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e27f7-d327-4634-86e3-62cd8aabed6c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.122857 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e27f7-d327-4634-86e3-62cd8aabed6c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.140098 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:52.180519 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cbda1-ad2d-42ac-8806-a045a60e69c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.181107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cbda1-ad2d-42ac-8806-a045a60e69c3 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.181383 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cbda1-ad2d-42ac-8806-a045a60e69c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.181650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cbda1-ad2d-42ac-8806-a045a60e69c3 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.182609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cbda1-ad2d-42ac-8806-a045a60e69c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.184619 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cbda1-ad2d-42ac-8806-a045a60e69c3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.259984 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1414/2831] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:18:52 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:18:52.264087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737e25f-f37e-4220-9f10-1b9414ec9724 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.276558 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737e25f-f37e-4220-9f10-1b9414ec9724 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.281113 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32565970-8e90-408f-b23b-8f9309cc9567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.281770 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32565970-8e90-408f-b23b-8f9309cc9567 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.282049 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32565970-8e90-408f-b23b-8f9309cc9567 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.282316 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32565970-8e90-408f-b23b-8f9309cc9567 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.283066 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32565970-8e90-408f-b23b-8f9309cc9567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.284640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32565970-8e90-408f-b23b-8f9309cc9567 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.295064 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1418/2832] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:52 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:52.299241 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cbda1-ad2d-42ac-8806-a045a60e69c3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.310402 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cbda1-ad2d-42ac-8806-a045a60e69c3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.315186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444752b9-83d6-4645-b8ea-cac20cc984a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.315879 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444752b9-83d6-4645-b8ea-cac20cc984a6 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.316162 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444752b9-83d6-4645-b8ea-cac20cc984a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.316438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444752b9-83d6-4645-b8ea-cac20cc984a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.317165 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444752b9-83d6-4645-b8ea-cac20cc984a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.319554 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444752b9-83d6-4645-b8ea-cac20cc984a6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.329516 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1415/2833] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:52 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:52.333633 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32565970-8e90-408f-b23b-8f9309cc9567 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.343870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32565970-8e90-408f-b23b-8f9309cc9567 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.348660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2f300-9389-45ed-908c-da54336e4313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.349539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2f300-9389-45ed-908c-da54336e4313 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.349880 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2f300-9389-45ed-908c-da54336e4313 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.350151 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2f300-9389-45ed-908c-da54336e4313 admin admin] PATH_INFO: `/v3/projects/df37299dc61246a2bd1ae4dd752d0bbc/users/f427f87464e04d1b8094235b5ef5f6bd/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.364367 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2f300-9389-45ed-908c-da54336e4313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df37299dc61246a2bd1ae4dd752d0bbc, user_id=f427f87464e04d1b8094235b5ef5f6bd, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.367553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2f300-9389-45ed-908c-da54336e4313 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.448452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1419/2834] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:18:52 2022] PUT /identity/v3/projects/df37299dc61246a2bd1ae4dd752d0bbc/users/f427f87464e04d1b8094235b5ef5f6bd/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:52.458724 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b3848-8078-4041-899c-62c080a57a4e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.459378 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b3848-8078-4041-899c-62c080a57a4e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.459659 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b3848-8078-4041-899c-62c080a57a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.487306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-004b3848-8078-4041-899c-62c080a57a4e None None] MFA Rules not processed for user `f427f87464e04d1b8094235b5ef5f6bd`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:52.488273 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-004b3848-8078-4041-899c-62c080a57a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:52.551511 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1416/2835] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:52.556730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2f300-9389-45ed-908c-da54336e4313 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.616454 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2f300-9389-45ed-908c-da54336e4313 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.620989 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8079e09-4df5-418a-9bf2-043d6447e10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.621639 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8079e09-4df5-418a-9bf2-043d6447e10e admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.621914 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8079e09-4df5-418a-9bf2-043d6447e10e admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.622180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8079e09-4df5-418a-9bf2-043d6447e10e admin admin] PATH_INFO: `/v3/projects` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.622886 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8079e09-4df5-418a-9bf2-043d6447e10e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.625100 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8079e09-4df5-418a-9bf2-043d6447e10e admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.723291 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1420/2836] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:18:52 2022] POST /identity/v3/projects => generated 376 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:52.727856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b3848-8078-4041-899c-62c080a57a4e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.738500 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b3848-8078-4041-899c-62c080a57a4e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.794668 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fc38a-ae86-4c07-8e47-c3c8355f2144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.795366 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fc38a-ae86-4c07-8e47-c3c8355f2144 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.795650 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fc38a-ae86-4c07-8e47-c3c8355f2144 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.795919 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fc38a-ae86-4c07-8e47-c3c8355f2144 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.796609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fc38a-ae86-4c07-8e47-c3c8355f2144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.798517 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fc38a-ae86-4c07-8e47-c3c8355f2144 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.874309 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1417/2837] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:18:52 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:18:52.878246 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8079e09-4df5-418a-9bf2-043d6447e10e admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.888498 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8079e09-4df5-418a-9bf2-043d6447e10e admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.893240 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047479f7-e181-4159-a540-5e3cb37c4323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.893901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047479f7-e181-4159-a540-5e3cb37c4323 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.894181 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047479f7-e181-4159-a540-5e3cb37c4323 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.894451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047479f7-e181-4159-a540-5e3cb37c4323 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.895260 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047479f7-e181-4159-a540-5e3cb37c4323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.898136 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047479f7-e181-4159-a540-5e3cb37c4323 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.899711 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:52.909120 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1421/2838] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:52 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:52.913987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fc38a-ae86-4c07-8e47-c3c8355f2144 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.924140 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fc38a-ae86-4c07-8e47-c3c8355f2144 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.928959 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d297a-f15b-435b-a7f7-e58afb92c549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.929886 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d297a-f15b-435b-a7f7-e58afb92c549 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.930128 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d297a-f15b-435b-a7f7-e58afb92c549 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.930401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d297a-f15b-435b-a7f7-e58afb92c549 admin admin] PATH_INFO: `/v3/projects/abf16d8e7de84c3b8930a8368c1ed9a7/users/454709c34527427f966ff10f5f71580b/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.945667 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d297a-f15b-435b-a7f7-e58afb92c549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abf16d8e7de84c3b8930a8368c1ed9a7, user_id=454709c34527427f966ff10f5f71580b, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.950372 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d297a-f15b-435b-a7f7-e58afb92c549 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.950924 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:53.025847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1418/2839] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:18:52 2022] PUT /identity/v3/projects/abf16d8e7de84c3b8930a8368c1ed9a7/users/454709c34527427f966ff10f5f71580b/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:53.030108 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047479f7-e181-4159-a540-5e3cb37c4323 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:53.040707 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047479f7-e181-4159-a540-5e3cb37c4323 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:53.096459 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdfb32-7466-430f-ae58-40a5ba76cef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:53.097162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdfb32-7466-430f-ae58-40a5ba76cef2 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.097451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdfb32-7466-430f-ae58-40a5ba76cef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.097729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdfb32-7466-430f-ae58-40a5ba76cef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.098451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdfb32-7466-430f-ae58-40a5ba76cef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:53.100730 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdfb32-7466-430f-ae58-40a5ba76cef2 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:53.111162 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1422/2840] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:18:53 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:18:53.115235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d297a-f15b-435b-a7f7-e58afb92c549 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:53.125589 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d297a-f15b-435b-a7f7-e58afb92c549 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:53.130405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa57e9f-fa85-4dbc-83be-84e490831e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:53.131329 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa57e9f-fa85-4dbc-83be-84e490831e11 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.131792 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa57e9f-fa85-4dbc-83be-84e490831e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.132081 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa57e9f-fa85-4dbc-83be-84e490831e11 admin admin] PATH_INFO: `/v3/projects/abf16d8e7de84c3b8930a8368c1ed9a7/users/454709c34527427f966ff10f5f71580b/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.134069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa57e9f-fa85-4dbc-83be-84e490831e11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abf16d8e7de84c3b8930a8368c1ed9a7, user_id=454709c34527427f966ff10f5f71580b, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:53.136729 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa57e9f-fa85-4dbc-83be-84e490831e11 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:53.148447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1419/2841] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:18:53 2022] PUT /identity/v3/projects/abf16d8e7de84c3b8930a8368c1ed9a7/users/454709c34527427f966ff10f5f71580b/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:18:53.159533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96437f51-0bed-4b72-b914-06d08e57e129 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.159823 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96437f51-0bed-4b72-b914-06d08e57e129 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.160092 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96437f51-0bed-4b72-b914-06d08e57e129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.186559 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-96437f51-0bed-4b72-b914-06d08e57e129 None None] MFA Rules not processed for user `454709c34527427f966ff10f5f71580b`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:18:53.187826 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-96437f51-0bed-4b72-b914-06d08e57e129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:53.253569 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1423/2842] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:18:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:18:53.265183 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa57e9f-fa85-4dbc-83be-84e490831e11 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:53.267326 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bfa57e9f-fa85-4dbc-83be-84e490831e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:53.324542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa57e9f-fa85-4dbc-83be-84e490831e11 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:53.328089 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322fea1-14f7-4bdd-a849-c5bda428279a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:53.328728 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322fea1-14f7-4bdd-a849-c5bda428279a service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:53.329006 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322fea1-14f7-4bdd-a849-c5bda428279a service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:53.329277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322fea1-14f7-4bdd-a849-c5bda428279a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:53.385107 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322fea1-14f7-4bdd-a849-c5bda428279a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:53.386541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322fea1-14f7-4bdd-a849-c5bda428279a service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:53.395232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1420/2843] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:55.420903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96437f51-0bed-4b72-b914-06d08e57e129 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:55.422463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-96437f51-0bed-4b72-b914-06d08e57e129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:55.435726 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96437f51-0bed-4b72-b914-06d08e57e129 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:55.439304 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7da0d-1d80-4431-bce4-187ffe107168 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:55.439994 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7da0d-1d80-4431-bce4-187ffe107168 service cinder] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:55.440279 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7da0d-1d80-4431-bce4-187ffe107168 service cinder] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:55.440557 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7da0d-1d80-4431-bce4-187ffe107168 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:55.499842 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7da0d-1d80-4431-bce4-187ffe107168 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:55.501267 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7da0d-1d80-4431-bce4-187ffe107168 service cinder] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:55.510672 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1424/2844] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:18:57.299771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322fea1-14f7-4bdd-a849-c5bda428279a service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:57.308825 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322fea1-14f7-4bdd-a849-c5bda428279a service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:57.312453 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a59ac-be5a-4839-ac0f-bc9775df325f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:57.313090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a59ac-be5a-4839-ac0f-bc9775df325f service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:57.313374 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a59ac-be5a-4839-ac0f-bc9775df325f service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:57.313649 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a59ac-be5a-4839-ac0f-bc9775df325f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:57.322850 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a59ac-be5a-4839-ac0f-bc9775df325f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:57.324277 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a59ac-be5a-4839-ac0f-bc9775df325f service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:57.334191 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1421/2845] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:18:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:02.118238 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7da0d-1d80-4431-bce4-187ffe107168 service cinder] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:02.140648 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=100 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:02.183517 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7da0d-1d80-4431-bce4-187ffe107168 service cinder] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:02.187976 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:02.188848 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:02.189130 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:02.189401 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] PATH_INFO: `/v3/users/c64dd84abd394dc5a93e774c15c6d141` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:02.190684 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c64dd84abd394dc5a93e774c15c6d141)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:02.192616 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:02.350104 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c64dd84abd394dc5a93e774c15c6d141'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:02.357005 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c64dd84abd394dc5a93e774c15c6d141'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:02.357675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c64dd84abd394dc5a93e774c15c6d141'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:02.369565 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c64dd84abd394dc5a93e774c15c6d141'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:02.445719 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c64dd84abd394dc5a93e774c15c6d141'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:02.451967 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1425/2846] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:19:02 2022] DELETE /identity/v3/users/c64dd84abd394dc5a93e774c15c6d141 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:02.456660 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b31ec-661e-4577-88de-25aba83a6cbd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:02.457148 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b31ec-661e-4577-88de-25aba83a6cbd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:02.457261 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b31ec-661e-4577-88de-25aba83a6cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:02.474759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e50b31ec-661e-4577-88de-25aba83a6cbd None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:02.478405 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e50b31ec-661e-4577-88de-25aba83a6cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:02.536254 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1422/2847] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:02.546373 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:02.547805 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:02.604941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e26da-2408-47e5-9bde-b14c23e768c0 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:02.608410 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcfcd2-51fc-474b-a597-6d0e7d415c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:02.609048 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcfcd2-51fc-474b-a597-6d0e7d415c25 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:02.609326 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcfcd2-51fc-474b-a597-6d0e7d415c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:02.609596 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcfcd2-51fc-474b-a597-6d0e7d415c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:02.619123 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dcfcd2-51fc-474b-a597-6d0e7d415c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:02.620525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dcfcd2-51fc-474b-a597-6d0e7d415c25 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:02.629689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1426/2848] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:03.038720 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b31ec-661e-4577-88de-25aba83a6cbd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:03.040301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e50b31ec-661e-4577-88de-25aba83a6cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:03.053972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b31ec-661e-4577-88de-25aba83a6cbd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:03.058337 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:03.059327 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:03.059579 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:03.059923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] PATH_INFO: `/v3/projects/483c00eb7e46472493eb9ee4baa16090` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:03.061156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=483c00eb7e46472493eb9ee4baa16090)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:03.063201 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:03.082382 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:03.174136 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '483c00eb7e46472493eb9ee4baa16090', 'request_id': 'req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:03.247784 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '483c00eb7e46472493eb9ee4baa16090', 'request_id': 'req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:03.254536 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '483c00eb7e46472493eb9ee4baa16090', 'request_id': 'req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:03.265271 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] The token cache is being invalidate because project 483c00eb7e46472493eb9ee4baa16090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:03.271074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:03.271074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:03.271074 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1423/2849] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:19:03 2022] DELETE /identity/v3/projects/483c00eb7e46472493eb9ee4baa16090 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:03.272787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcfcd2-51fc-474b-a597-6d0e7d415c25 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:03.282287 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcfcd2-51fc-474b-a597-6d0e7d415c25 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:03.320542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:03.338158 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:03.339054 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:03.339329 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:03.339598 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] PATH_INFO: `/v3/users/f427f87464e04d1b8094235b5ef5f6bd` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:03.340753 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f427f87464e04d1b8094235b5ef5f6bd)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:03.342635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:03.516918 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f427f87464e04d1b8094235b5ef5f6bd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:03.523640 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f427f87464e04d1b8094235b5ef5f6bd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:03.524294 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f427f87464e04d1b8094235b5ef5f6bd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:03.536030 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f427f87464e04d1b8094235b5ef5f6bd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:03.630112 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f427f87464e04d1b8094235b5ef5f6bd'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:03.637193 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1427/2850] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:19:03 2022] DELETE /identity/v3/users/f427f87464e04d1b8094235b5ef5f6bd => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:04.042008 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:04.043609 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:04.057178 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f0a26-cc8e-48c0-8531-5a09d07bd8a4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:04.116314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:04.117169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:04.117445 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:04.117731 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] PATH_INFO: `/v3/projects/df37299dc61246a2bd1ae4dd752d0bbc` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:04.119020 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=df37299dc61246a2bd1ae4dd752d0bbc)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:04.120911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:04.253240 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df37299dc61246a2bd1ae4dd752d0bbc', 'request_id': 'req-d4bda0f8-93ea-4246-8b14-502474714c9b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:04.327663 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df37299dc61246a2bd1ae4dd752d0bbc', 'request_id': 'req-d4bda0f8-93ea-4246-8b14-502474714c9b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:04.334993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df37299dc61246a2bd1ae4dd752d0bbc', 'request_id': 'req-d4bda0f8-93ea-4246-8b14-502474714c9b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:04.345193 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] The token cache is being invalidate because project df37299dc61246a2bd1ae4dd752d0bbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:04.345477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:04.346105 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:04.348029 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1424/2851] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:19:04 2022] DELETE /identity/v3/projects/df37299dc61246a2bd1ae4dd752d0bbc => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:04.353009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:04.354409 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:04.367188 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6cf18-8114-463d-9ef7-8f8e625ae790 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:04.422564 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979eefa-7ebb-41a8-93e6-cd8325a9e486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:04.423529 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979eefa-7ebb-41a8-93e6-cd8325a9e486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:04.423806 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979eefa-7ebb-41a8-93e6-cd8325a9e486 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:04.424076 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979eefa-7ebb-41a8-93e6-cd8325a9e486 admin admin] PATH_INFO: `/v3/users/454709c34527427f966ff10f5f71580b` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:04.425253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979eefa-7ebb-41a8-93e6-cd8325a9e486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=454709c34527427f966ff10f5f71580b)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:04.427180 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979eefa-7ebb-41a8-93e6-cd8325a9e486 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:04.638525 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-e979eefa-7ebb-41a8-93e6-cd8325a9e486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454709c34527427f966ff10f5f71580b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:04.645306 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-e979eefa-7ebb-41a8-93e6-cd8325a9e486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454709c34527427f966ff10f5f71580b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:04.645959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-e979eefa-7ebb-41a8-93e6-cd8325a9e486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454709c34527427f966ff10f5f71580b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:04.650434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:04.658333 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-e979eefa-7ebb-41a8-93e6-cd8325a9e486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454709c34527427f966ff10f5f71580b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:04.710804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-e979eefa-7ebb-41a8-93e6-cd8325a9e486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454709c34527427f966ff10f5f71580b'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:04.717135 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1428/2852] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:19:04 2022] DELETE /identity/v3/users/454709c34527427f966ff10f5f71580b => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:05.150847 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:05.152291 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:05.165358 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bda0f8-93ea-4246-8b14-502474714c9b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:05.198733 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:05.222731 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:05.223626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:05.223911 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:05.224197 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00 admin admin] PATH_INFO: `/v3/projects/abf16d8e7de84c3b8930a8368c1ed9a7` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:05.225524 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abf16d8e7de84c3b8930a8368c1ed9a7)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:05.227491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:05.356669 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abf16d8e7de84c3b8930a8368c1ed9a7', 'request_id': 'req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:05.445436 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abf16d8e7de84c3b8930a8368c1ed9a7', 'request_id': 'req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:05.452032 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abf16d8e7de84c3b8930a8368c1ed9a7', 'request_id': 'req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:05.462950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00 admin admin] The token cache is being invalidate because project abf16d8e7de84c3b8930a8368c1ed9a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:05.463236 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:05.463859 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9c7e65da-f9e0-4ba7-a7fe-97fd03afce00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:05.465377 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1425/2853] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:19:05 2022] DELETE /identity/v3/projects/abf16d8e7de84c3b8930a8368c1ed9a7 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:05.470254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d58832-2057-44ff-acbe-a3a0c2db662a None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:05.470721 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d58832-2057-44ff-acbe-a3a0c2db662a None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:05.470915 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d58832-2057-44ff-acbe-a3a0c2db662a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:05.489385 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-c6d58832-2057-44ff-acbe-a3a0c2db662a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:05.490645 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-c6d58832-2057-44ff-acbe-a3a0c2db662a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:05.548919 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1429/2854] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:19:05.557156 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce29b75-9e85-4b53-8824-32a4db4c676b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:05.557715 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce29b75-9e85-4b53-8824-32a4db4c676b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:05.557988 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce29b75-9e85-4b53-8824-32a4db4c676b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:05.576548 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1ce29b75-9e85-4b53-8824-32a4db4c676b None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:05.577744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1ce29b75-9e85-4b53-8824-32a4db4c676b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:05.585601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1426/2855] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:05.589704 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d58832-2057-44ff-acbe-a3a0c2db662a None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:05.599128 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d58832-2057-44ff-acbe-a3a0c2db662a None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:05.603668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ce17b-33c0-4681-bf3e-8ec5ed37516f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:05.604314 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ce17b-33c0-4681-bf3e-8ec5ed37516f admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:05.604591 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ce17b-33c0-4681-bf3e-8ec5ed37516f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:05.604861 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ce17b-33c0-4681-bf3e-8ec5ed37516f admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:05.605571 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ce17b-33c0-4681-bf3e-8ec5ed37516f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:19:05.605896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ce17b-33c0-4681-bf3e-8ec5ed37516f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:05.607522 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ce17b-33c0-4681-bf3e-8ec5ed37516f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:05.619441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1430/2856] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:19:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:19:05.623635 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce29b75-9e85-4b53-8824-32a4db4c676b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:05.633149 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce29b75-9e85-4b53-8824-32a4db4c676b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:05.637656 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd097b1-2945-4494-be34-15a572855a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:05.638288 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd097b1-2945-4494-be34-15a572855a63 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:05.638562 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd097b1-2945-4494-be34-15a572855a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:05.638855 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd097b1-2945-4494-be34-15a572855a63 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:05.639533 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd097b1-2945-4494-be34-15a572855a63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:05.641425 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd097b1-2945-4494-be34-15a572855a63 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:05.752154 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1427/2857] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:19:05 2022] POST /identity/v3/projects => generated 374 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:19:05.756995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ce17b-33c0-4681-bf3e-8ec5ed37516f admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:05.766980 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ce17b-33c0-4681-bf3e-8ec5ed37516f admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:05.824343 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82074f7a-f6ad-4b92-98d0-6b3af8ae41e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:05.825015 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82074f7a-f6ad-4b92-98d0-6b3af8ae41e7 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:05.825295 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82074f7a-f6ad-4b92-98d0-6b3af8ae41e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:05.825570 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82074f7a-f6ad-4b92-98d0-6b3af8ae41e7 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:05.826251 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82074f7a-f6ad-4b92-98d0-6b3af8ae41e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:05.828177 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82074f7a-f6ad-4b92-98d0-6b3af8ae41e7 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:05.939673 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1431/2858] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:19:05 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:05.944496 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd097b1-2945-4494-be34-15a572855a63 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:05.954750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd097b1-2945-4494-be34-15a572855a63 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:05.960090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a3e9e-9635-414f-8952-12e9cb724749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:05.960754 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a3e9e-9635-414f-8952-12e9cb724749 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:05.961039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a3e9e-9635-414f-8952-12e9cb724749 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:05.961319 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a3e9e-9635-414f-8952-12e9cb724749 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:05.962044 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a3e9e-9635-414f-8952-12e9cb724749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:05.963679 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a3e9e-9635-414f-8952-12e9cb724749 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:05.973987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1428/2859] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:19:05 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:05.978148 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82074f7a-f6ad-4b92-98d0-6b3af8ae41e7 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:05.988072 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82074f7a-f6ad-4b92-98d0-6b3af8ae41e7 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:05.993022 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39657477-550c-4050-8e31-407ca96e046f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:05.993685 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39657477-550c-4050-8e31-407ca96e046f admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:05.993968 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39657477-550c-4050-8e31-407ca96e046f admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:05.994247 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39657477-550c-4050-8e31-407ca96e046f admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:05.995101 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39657477-550c-4050-8e31-407ca96e046f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:05.996686 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39657477-550c-4050-8e31-407ca96e046f admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:06.007278 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1432/2860] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:19:05 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:06.011364 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a3e9e-9635-414f-8952-12e9cb724749 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:06.021232 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a3e9e-9635-414f-8952-12e9cb724749 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:06.026368 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841ae4d-6ab6-497c-9982-55f3a48bee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:06.027272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841ae4d-6ab6-497c-9982-55f3a48bee68 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:06.027550 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841ae4d-6ab6-497c-9982-55f3a48bee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:06.027818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841ae4d-6ab6-497c-9982-55f3a48bee68 admin admin] PATH_INFO: `/v3/projects/eb3a382173dd4b9787fdfdf77c7d75e7/users/ae5e1ef8b78449c0a345565c48c71b20/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:06.042299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841ae4d-6ab6-497c-9982-55f3a48bee68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb3a382173dd4b9787fdfdf77c7d75e7, user_id=ae5e1ef8b78449c0a345565c48c71b20, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:06.044950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841ae4d-6ab6-497c-9982-55f3a48bee68 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:06.134371 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1429/2861] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:19:06 2022] PUT /identity/v3/projects/eb3a382173dd4b9787fdfdf77c7d75e7/users/ae5e1ef8b78449c0a345565c48c71b20/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:06.144696 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5b290-b97a-4d31-9b53-afecf4db75e1 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:06.144987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5b290-b97a-4d31-9b53-afecf4db75e1 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:06.145296 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5b290-b97a-4d31-9b53-afecf4db75e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:06.172331 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-60d5b290-b97a-4d31-9b53-afecf4db75e1 None None] MFA Rules not processed for user `ae5e1ef8b78449c0a345565c48c71b20`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:06.173532 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-60d5b290-b97a-4d31-9b53-afecf4db75e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:06.206021 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:06.238313 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1433/2862] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:06.249706 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841ae4d-6ab6-497c-9982-55f3a48bee68 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:06.251478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2841ae4d-6ab6-497c-9982-55f3a48bee68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:06.308479 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841ae4d-6ab6-497c-9982-55f3a48bee68 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:06.312186 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ebfe0-7914-4926-8cec-2d7904fbbde9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:06.312834 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ebfe0-7914-4926-8cec-2d7904fbbde9 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:06.313116 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ebfe0-7914-4926-8cec-2d7904fbbde9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:06.313388 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ebfe0-7914-4926-8cec-2d7904fbbde9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:06.322218 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ebfe0-7914-4926-8cec-2d7904fbbde9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:06.323661 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ebfe0-7914-4926-8cec-2d7904fbbde9 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:06.332240 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1430/2863] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:19:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:06.356535 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5b290-b97a-4d31-9b53-afecf4db75e1 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:06.419695 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5b290-b97a-4d31-9b53-afecf4db75e1 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:06.424234 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:06.425090 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:06.425368 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:06.425635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] PATH_INFO: `/v3/users/ae5e1ef8b78449c0a345565c48c71b20` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:06.426803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae5e1ef8b78449c0a345565c48c71b20)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:06.429354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:06.651553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae5e1ef8b78449c0a345565c48c71b20'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:06.658307 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae5e1ef8b78449c0a345565c48c71b20'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:06.659002 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae5e1ef8b78449c0a345565c48c71b20'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:06.671066 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae5e1ef8b78449c0a345565c48c71b20'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:06.738873 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae5e1ef8b78449c0a345565c48c71b20'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:06.745660 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1434/2864] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:19:06 2022] DELETE /identity/v3/users/ae5e1ef8b78449c0a345565c48c71b20 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:06.750184 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9a3a1-f44e-40ed-9ecd-8a7e70b1b279 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:06.750491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9a3a1-f44e-40ed-9ecd-8a7e70b1b279 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:06.750773 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9a3a1-f44e-40ed-9ecd-8a7e70b1b279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:06.757373 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:06.767967 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ced9a3a1-f44e-40ed-9ecd-8a7e70b1b279 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:06.769172 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ced9a3a1-f44e-40ed-9ecd-8a7e70b1b279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:06.829086 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1431/2865] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:06.839391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:06.840781 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:06.899896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924f34a-eeed-4c75-b84c-9a80be14bafa admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:06.903300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede7b4d-b2db-4ef1-b403-933d5001211f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:06.903952 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede7b4d-b2db-4ef1-b403-933d5001211f service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:06.904250 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede7b4d-b2db-4ef1-b403-933d5001211f service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:06.904510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede7b4d-b2db-4ef1-b403-933d5001211f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:06.911487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:06.913981 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede7b4d-b2db-4ef1-b403-933d5001211f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:06.915455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede7b4d-b2db-4ef1-b403-933d5001211f service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:06.924945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1435/2866] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:07.278306 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9a3a1-f44e-40ed-9ecd-8a7e70b1b279 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:07.279791 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ced9a3a1-f44e-40ed-9ecd-8a7e70b1b279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:07.293418 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9a3a1-f44e-40ed-9ecd-8a7e70b1b279 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:07.298190 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823fcd1-a90a-4823-8a8d-cae7152e161b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:07.299088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823fcd1-a90a-4823-8a8d-cae7152e161b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:07.299390 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823fcd1-a90a-4823-8a8d-cae7152e161b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:07.299673 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823fcd1-a90a-4823-8a8d-cae7152e161b admin admin] PATH_INFO: `/v3/projects/eb3a382173dd4b9787fdfdf77c7d75e7` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:07.300827 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823fcd1-a90a-4823-8a8d-cae7152e161b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb3a382173dd4b9787fdfdf77c7d75e7)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:07.305675 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823fcd1-a90a-4823-8a8d-cae7152e161b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:07.414785 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9823fcd1-a90a-4823-8a8d-cae7152e161b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb3a382173dd4b9787fdfdf77c7d75e7', 'request_id': 'req-9823fcd1-a90a-4823-8a8d-cae7152e161b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:07.481680 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9823fcd1-a90a-4823-8a8d-cae7152e161b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb3a382173dd4b9787fdfdf77c7d75e7', 'request_id': 'req-9823fcd1-a90a-4823-8a8d-cae7152e161b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:07.488936 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9823fcd1-a90a-4823-8a8d-cae7152e161b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb3a382173dd4b9787fdfdf77c7d75e7', 'request_id': 'req-9823fcd1-a90a-4823-8a8d-cae7152e161b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:07.500299 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9823fcd1-a90a-4823-8a8d-cae7152e161b admin admin] The token cache is being invalidate because project eb3a382173dd4b9787fdfdf77c7d75e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:07.500591 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9823fcd1-a90a-4823-8a8d-cae7152e161b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:07.501293 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9823fcd1-a90a-4823-8a8d-cae7152e161b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:07.502842 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1432/2867] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:19:07 2022] DELETE /identity/v3/projects/eb3a382173dd4b9787fdfdf77c7d75e7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:07.508452 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f083dbd-cb0e-4f5f-93e6-9f1cdf4ce2bc None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:07.508737 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f083dbd-cb0e-4f5f-93e6-9f1cdf4ce2bc None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:07.509097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f083dbd-cb0e-4f5f-93e6-9f1cdf4ce2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:07.527553 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-5f083dbd-cb0e-4f5f-93e6-9f1cdf4ce2bc None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:07.528736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5f083dbd-cb0e-4f5f-93e6-9f1cdf4ce2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:07.587474 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1436/2868] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:07.647469 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64742f-8e60-4fe8-ac72-66fc6ae1c52a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:07.647759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64742f-8e60-4fe8-ac72-66fc6ae1c52a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:07.648039 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64742f-8e60-4fe8-ac72-66fc6ae1c52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:07.667108 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9e64742f-8e60-4fe8-ac72-66fc6ae1c52a None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:07.668242 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9e64742f-8e60-4fe8-ac72-66fc6ae1c52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:07.676027 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1433/2869] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:07.680997 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f083dbd-cb0e-4f5f-93e6-9f1cdf4ce2bc None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:07.690995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f083dbd-cb0e-4f5f-93e6-9f1cdf4ce2bc None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:07.696512 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba13366-7cb7-4328-8508-b7785823ac40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:07.697169 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba13366-7cb7-4328-8508-b7785823ac40 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:07.697455 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba13366-7cb7-4328-8508-b7785823ac40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:07.697734 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba13366-7cb7-4328-8508-b7785823ac40 admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:07.698456 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba13366-7cb7-4328-8508-b7785823ac40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:19:07.698765 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba13366-7cb7-4328-8508-b7785823ac40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:07.700428 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba13366-7cb7-4328-8508-b7785823ac40 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:07.713146 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1437/2870] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:19:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:19:07.717407 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64742f-8e60-4fe8-ac72-66fc6ae1c52a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:07.726913 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64742f-8e60-4fe8-ac72-66fc6ae1c52a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:07.731573 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853739e-54ec-4116-a76c-ec965a05a7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:07.732222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853739e-54ec-4116-a76c-ec965a05a7d4 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:07.732503 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853739e-54ec-4116-a76c-ec965a05a7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:07.732774 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853739e-54ec-4116-a76c-ec965a05a7d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:07.733491 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853739e-54ec-4116-a76c-ec965a05a7d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:07.735454 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853739e-54ec-4116-a76c-ec965a05a7d4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:07.819652 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1434/2871] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:19:07 2022] POST /identity/v3/projects => generated 398 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:07.823948 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba13366-7cb7-4328-8508-b7785823ac40 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:07.834053 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba13366-7cb7-4328-8508-b7785823ac40 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:07.890337 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35a4fa-019d-4c79-b164-99caa7ed5710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:07.891066 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35a4fa-019d-4c79-b164-99caa7ed5710 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:07.891320 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35a4fa-019d-4c79-b164-99caa7ed5710 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:07.891611 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35a4fa-019d-4c79-b164-99caa7ed5710 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:07.892297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35a4fa-019d-4c79-b164-99caa7ed5710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:07.894191 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35a4fa-019d-4c79-b164-99caa7ed5710 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:07.966354 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1438/2872] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:19:07 2022] POST /identity/v3/users => generated 300 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:07.970386 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853739e-54ec-4116-a76c-ec965a05a7d4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:07.980954 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853739e-54ec-4116-a76c-ec965a05a7d4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:07.985630 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b26460-1226-4e65-8f01-6b68210504a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:07.986287 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26460-1226-4e65-8f01-6b68210504a0 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:07.986569 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26460-1226-4e65-8f01-6b68210504a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:07.986870 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26460-1226-4e65-8f01-6b68210504a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:07.987601 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b26460-1226-4e65-8f01-6b68210504a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:07.989196 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b26460-1226-4e65-8f01-6b68210504a0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:08.000687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1435/2873] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:19:07 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:08.004533 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35a4fa-019d-4c79-b164-99caa7ed5710 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:08.015936 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35a4fa-019d-4c79-b164-99caa7ed5710 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:08.020668 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a4858-8980-4255-be7c-ca3708e29844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:08.022019 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a4858-8980-4255-be7c-ca3708e29844 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:08.022305 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a4858-8980-4255-be7c-ca3708e29844 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:08.022582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a4858-8980-4255-be7c-ca3708e29844 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:08.023344 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a4858-8980-4255-be7c-ca3708e29844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:08.024949 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a4858-8980-4255-be7c-ca3708e29844 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:08.034899 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1439/2874] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:19:08 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:19:08.039918 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b26460-1226-4e65-8f01-6b68210504a0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:08.049748 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b26460-1226-4e65-8f01-6b68210504a0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:08.054332 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834517f-43f3-4cd7-89db-82f1a316409e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:08.055248 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834517f-43f3-4cd7-89db-82f1a316409e admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:08.055530 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834517f-43f3-4cd7-89db-82f1a316409e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:08.055802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834517f-43f3-4cd7-89db-82f1a316409e admin admin] PATH_INFO: `/v3/projects/2f3b8a95e95a4eb496efbb234905f2cd/users/c78f4597bc294c0fb1e87f7da629a921/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:08.070718 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834517f-43f3-4cd7-89db-82f1a316409e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f3b8a95e95a4eb496efbb234905f2cd, user_id=c78f4597bc294c0fb1e87f7da629a921, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:08.073398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834517f-43f3-4cd7-89db-82f1a316409e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:08.179903 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1436/2875] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:19:08 2022] PUT /identity/v3/projects/2f3b8a95e95a4eb496efbb234905f2cd/users/c78f4597bc294c0fb1e87f7da629a921/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:19:08.189995 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db590612-eb1e-4592-b966-d3c7a814ce20 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:08.190229 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db590612-eb1e-4592-b966-d3c7a814ce20 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:08.190510 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db590612-eb1e-4592-b966-d3c7a814ce20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:08.217580 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-db590612-eb1e-4592-b966-d3c7a814ce20 None None] MFA Rules not processed for user `c78f4597bc294c0fb1e87f7da629a921`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:08.218770 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-db590612-eb1e-4592-b966-d3c7a814ce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:08.264187 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:08.281059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1440/2876] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:08 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:08.293615 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834517f-43f3-4cd7-89db-82f1a316409e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:08.295338 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2834517f-43f3-4cd7-89db-82f1a316409e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:08.323565 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:08.352610 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834517f-43f3-4cd7-89db-82f1a316409e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:08.356395 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a06a11-cf83-49c8-8a2d-47f7d8f43211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25d85d02abed40199dd2515ecf84b7fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:08.357034 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a06a11-cf83-49c8-8a2d-47f7d8f43211 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:08.357317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a06a11-cf83-49c8-8a2d-47f7d8f43211 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:08.357587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a06a11-cf83-49c8-8a2d-47f7d8f43211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:08.366961 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a06a11-cf83-49c8-8a2d-47f7d8f43211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:08.368463 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a06a11-cf83-49c8-8a2d-47f7d8f43211 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:08.377397 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1437/2877] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:19:08 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:17.372058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db590612-eb1e-4592-b966-d3c7a814ce20 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.434723 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db590612-eb1e-4592-b966-d3c7a814ce20 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.439254 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.440119 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.440399 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.440673 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] PATH_INFO: `/v3/users/c78f4597bc294c0fb1e87f7da629a921` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.441836 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c78f4597bc294c0fb1e87f7da629a921)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.443810 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.629954 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c78f4597bc294c0fb1e87f7da629a921'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:17.636941 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c78f4597bc294c0fb1e87f7da629a921'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:17.637604 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c78f4597bc294c0fb1e87f7da629a921'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:17.649769 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c78f4597bc294c0fb1e87f7da629a921'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:17.713601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c78f4597bc294c0fb1e87f7da629a921'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:17.720127 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1441/2878] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:19:17 2022] DELETE /identity/v3/users/c78f4597bc294c0fb1e87f7da629a921 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:17.724626 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33385551-5b01-4d8f-8f79-578a92af0a71 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.725171 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33385551-5b01-4d8f-8f79-578a92af0a71 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.725300 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33385551-5b01-4d8f-8f79-578a92af0a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.742963 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-33385551-5b01-4d8f-8f79-578a92af0a71 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:17.744259 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-33385551-5b01-4d8f-8f79-578a92af0a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:17.802406 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1438/2879] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:17.812485 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:17.813959 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:17.869582 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a5011-f509-4acc-a6dd-dd510cb6a7fa admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:17.873167 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ab815-fecf-4c57-8fe3-c13a168ed432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:17.873804 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ab815-fecf-4c57-8fe3-c13a168ed432 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:17.874082 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ab815-fecf-4c57-8fe3-c13a168ed432 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:17.874351 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ab815-fecf-4c57-8fe3-c13a168ed432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:17.883811 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ab815-fecf-4c57-8fe3-c13a168ed432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:17.885920 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ab815-fecf-4c57-8fe3-c13a168ed432 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:17.895282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1442/2880] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:18.294495 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33385551-5b01-4d8f-8f79-578a92af0a71 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:18.296069 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-33385551-5b01-4d8f-8f79-578a92af0a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:18.310950 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33385551-5b01-4d8f-8f79-578a92af0a71 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:18.315687 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d75b1-7936-4713-8402-a3b63b7df5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:18.317249 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d75b1-7936-4713-8402-a3b63b7df5a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:18.317532 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d75b1-7936-4713-8402-a3b63b7df5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:18.317894 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d75b1-7936-4713-8402-a3b63b7df5a9 admin admin] PATH_INFO: `/v3/projects/2f3b8a95e95a4eb496efbb234905f2cd` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:18.319167 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d75b1-7936-4713-8402-a3b63b7df5a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f3b8a95e95a4eb496efbb234905f2cd)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:18.322040 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d75b1-7936-4713-8402-a3b63b7df5a9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:18.424121 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d95d75b1-7936-4713-8402-a3b63b7df5a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f3b8a95e95a4eb496efbb234905f2cd', 'request_id': 'req-d95d75b1-7936-4713-8402-a3b63b7df5a9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:18.498655 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d95d75b1-7936-4713-8402-a3b63b7df5a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f3b8a95e95a4eb496efbb234905f2cd', 'request_id': 'req-d95d75b1-7936-4713-8402-a3b63b7df5a9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:18.505604 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d95d75b1-7936-4713-8402-a3b63b7df5a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f3b8a95e95a4eb496efbb234905f2cd', 'request_id': 'req-d95d75b1-7936-4713-8402-a3b63b7df5a9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:18.516804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d95d75b1-7936-4713-8402-a3b63b7df5a9 admin admin] The token cache is being invalidate because project 2f3b8a95e95a4eb496efbb234905f2cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:19:18.517090 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d95d75b1-7936-4713-8402-a3b63b7df5a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:18.517726 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d95d75b1-7936-4713-8402-a3b63b7df5a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:19:18.519298 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1439/2881] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:19:18 2022] DELETE /identity/v3/projects/2f3b8a95e95a4eb496efbb234905f2cd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:18.524255 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c66d73-3baa-4e05-a9b7-711e5526bc6e None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:18.524755 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c66d73-3baa-4e05-a9b7-711e5526bc6e None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:18.524862 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c66d73-3baa-4e05-a9b7-711e5526bc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:18.542855 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-e8c66d73-3baa-4e05-a9b7-711e5526bc6e None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:18.544057 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-e8c66d73-3baa-4e05-a9b7-711e5526bc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:18.552641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:18.604926 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1443/2882] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:18.613968 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f45f1-f9e9-4397-95c8-64f2d19f9742 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:18.614256 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f45f1-f9e9-4397-95c8-64f2d19f9742 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:18.614539 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f45f1-f9e9-4397-95c8-64f2d19f9742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:18.634110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f49f45f1-f9e9-4397-95c8-64f2d19f9742 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:18.635527 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f49f45f1-f9e9-4397-95c8-64f2d19f9742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:18.645124 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1440/2883] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:18.648058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c66d73-3baa-4e05-a9b7-711e5526bc6e None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:18.658176 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c66d73-3baa-4e05-a9b7-711e5526bc6e None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:18.662614 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d461e9-1b96-4284-b4b2-68978b95befa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:18.663391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d461e9-1b96-4284-b4b2-68978b95befa admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:18.663835 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d461e9-1b96-4284-b4b2-68978b95befa admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:18.664115 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d461e9-1b96-4284-b4b2-68978b95befa admin admin] PATH_INFO: `/v3/domains` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:18.665358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d461e9-1b96-4284-b4b2-68978b95befa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92082) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:19:18.665748 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d461e9-1b96-4284-b4b2-68978b95befa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:18.667379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d461e9-1b96-4284-b4b2-68978b95befa admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:18.680780 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1444/2884] 10.222.0.121 () {58 vars in 1157 bytes} [Thu May 26 18:19:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:19:18.685119 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f45f1-f9e9-4397-95c8-64f2d19f9742 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:18.694260 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f45f1-f9e9-4397-95c8-64f2d19f9742 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:18.699215 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ca3bd-a5cd-4d26-80e0-94703eee1396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:18.699856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ca3bd-a5cd-4d26-80e0-94703eee1396 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:18.700133 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ca3bd-a5cd-4d26-80e0-94703eee1396 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:18.700399 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ca3bd-a5cd-4d26-80e0-94703eee1396 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:18.701091 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ca3bd-a5cd-4d26-80e0-94703eee1396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:18.703094 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ca3bd-a5cd-4d26-80e0-94703eee1396 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:18.818410 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1441/2885] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:19:18 2022] POST /identity/v3/projects => generated 386 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:18.822896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d461e9-1b96-4284-b4b2-68978b95befa admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:18.833058 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d461e9-1b96-4284-b4b2-68978b95befa admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:18.890319 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b16f2-c5f2-4554-9272-8a2bca064721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:18.891017 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b16f2-c5f2-4554-9272-8a2bca064721 admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:18.891300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b16f2-c5f2-4554-9272-8a2bca064721 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:18.891574 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b16f2-c5f2-4554-9272-8a2bca064721 admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:18.892256 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b16f2-c5f2-4554-9272-8a2bca064721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:18.894131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b16f2-c5f2-4554-9272-8a2bca064721 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:18.971577 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1445/2886] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:19:18 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:18.975672 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ca3bd-a5cd-4d26-80e0-94703eee1396 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:18.986802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ca3bd-a5cd-4d26-80e0-94703eee1396 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:18.991611 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b6885-d614-4788-88bd-9cfb76c1b1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:18.992267 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b6885-d614-4788-88bd-9cfb76c1b1cb admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:18.992567 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b6885-d614-4788-88bd-9cfb76c1b1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:18.993587 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b6885-d614-4788-88bd-9cfb76c1b1cb admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:18.994401 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b6885-d614-4788-88bd-9cfb76c1b1cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:18.996093 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b6885-d614-4788-88bd-9cfb76c1b1cb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:19.006551 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1442/2887] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:19:18 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:19.010641 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b16f2-c5f2-4554-9272-8a2bca064721 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:19.020924 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b16f2-c5f2-4554-9272-8a2bca064721 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:19.026434 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b48cb1-6b34-4725-b5ed-630a0f209ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:19.027131 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b48cb1-6b34-4725-b5ed-630a0f209ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:19.027408 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b48cb1-6b34-4725-b5ed-630a0f209ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:19.027678 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b48cb1-6b34-4725-b5ed-630a0f209ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:19.028393 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b48cb1-6b34-4725-b5ed-630a0f209ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:19.029961 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b48cb1-6b34-4725-b5ed-630a0f209ca6 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:19.040901 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1446/2888] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:19:19 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:19.045066 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b6885-d614-4788-88bd-9cfb76c1b1cb admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:19.054992 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b6885-d614-4788-88bd-9cfb76c1b1cb admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:19.059856 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23407de-73fc-4b4f-9e7d-ff8d945e1078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:19.060725 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23407de-73fc-4b4f-9e7d-ff8d945e1078 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:19.061001 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23407de-73fc-4b4f-9e7d-ff8d945e1078 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:19.061272 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23407de-73fc-4b4f-9e7d-ff8d945e1078 admin admin] PATH_INFO: `/v3/projects/5950d9e178c2442cb0c62214d1eee60e/users/e88e293efd2442d586abc01e621ac415/roles/e87c07d8d70d46d89a2c4f7246edeff6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:19.069438 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:19.076409 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23407de-73fc-4b4f-9e7d-ff8d945e1078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5950d9e178c2442cb0c62214d1eee60e, user_id=e88e293efd2442d586abc01e621ac415, role_id=e87c07d8d70d46d89a2c4f7246edeff6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:19.081781 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23407de-73fc-4b4f-9e7d-ff8d945e1078 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:19.151631 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1443/2889] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:19:19 2022] PUT /identity/v3/projects/5950d9e178c2442cb0c62214d1eee60e/users/e88e293efd2442d586abc01e621ac415/roles/e87c07d8d70d46d89a2c4f7246edeff6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:19.156739 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835b843-0a3f-4d02-b3de-eb93810ddba7 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:19.157009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835b843-0a3f-4d02-b3de-eb93810ddba7 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:19.157289 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835b843-0a3f-4d02-b3de-eb93810ddba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:19.176903 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-0835b843-0a3f-4d02-b3de-eb93810ddba7 None None] MFA Rules not processed for user `9845c1dd09f34a0aa117c5e212e20304`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:19.178105 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-0835b843-0a3f-4d02-b3de-eb93810ddba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:19.236839 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1447/2890] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:19.246542 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23407de-73fc-4b4f-9e7d-ff8d945e1078 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:19.248747 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c23407de-73fc-4b4f-9e7d-ff8d945e1078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:19.304150 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23407de-73fc-4b4f-9e7d-ff8d945e1078 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:19.308146 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7427f4-4a19-41a1-b901-1fc279e86acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:19.308789 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7427f4-4a19-41a1-b901-1fc279e86acb service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:19.309070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7427f4-4a19-41a1-b901-1fc279e86acb service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:19.309349 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7427f4-4a19-41a1-b901-1fc279e86acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:19.318221 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7427f4-4a19-41a1-b901-1fc279e86acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:19.319681 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7427f4-4a19-41a1-b901-1fc279e86acb service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:19.329169 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1444/2891] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:19:24.452384 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7d2b1-1783-4d97-bdbb-e88f2cc6dbb3 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:24.452840 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7d2b1-1783-4d97-bdbb-e88f2cc6dbb3 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:24.452929 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7d2b1-1783-4d97-bdbb-e88f2cc6dbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:24.501630 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-a9e7d2b1-1783-4d97-bdbb-e88f2cc6dbb3 None None] MFA Rules not processed for user `e88e293efd2442d586abc01e621ac415`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:24.502945 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a9e7d2b1-1783-4d97-bdbb-e88f2cc6dbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:24.512379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:24.593009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1448/2892] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:24.597450 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7427f4-4a19-41a1-b901-1fc279e86acb service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:24.609545 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7427f4-4a19-41a1-b901-1fc279e86acb service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:24.614456 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0556c39-9f25-49b8-a9cc-8541f2b0bfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:24.615285 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0556c39-9f25-49b8-a9cc-8541f2b0bfba admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:24.615705 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0556c39-9f25-49b8-a9cc-8541f2b0bfba admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:24.616070 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0556c39-9f25-49b8-a9cc-8541f2b0bfba admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:24.616916 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0556c39-9f25-49b8-a9cc-8541f2b0bfba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:24.619808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0556c39-9f25-49b8-a9cc-8541f2b0bfba admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:24.709416 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1445/2893] 10.222.0.121 () {60 vars in 1157 bytes} [Thu May 26 18:19:24 2022] POST /identity/v3/projects => generated 386 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:24.710661 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7d2b1-1783-4d97-bdbb-e88f2cc6dbb3 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:24.723271 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7d2b1-1783-4d97-bdbb-e88f2cc6dbb3 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:24.785729 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d94a8-1371-48c1-9e54-cf5fccb8a0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:24.786411 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d94a8-1371-48c1-9e54-cf5fccb8a0ff admin admin] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:24.786689 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d94a8-1371-48c1-9e54-cf5fccb8a0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:24.786992 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d94a8-1371-48c1-9e54-cf5fccb8a0ff admin admin] PATH_INFO: `/v3/users` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:24.787679 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d94a8-1371-48c1-9e54-cf5fccb8a0ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:24.789587 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d94a8-1371-48c1-9e54-cf5fccb8a0ff admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:24.884404 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1449/2894] 10.222.0.121 () {60 vars in 1148 bytes} [Thu May 26 18:19:24 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:19:24.887560 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0556c39-9f25-49b8-a9cc-8541f2b0bfba admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:24.909317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0556c39-9f25-49b8-a9cc-8541f2b0bfba admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:24.917952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdf7e1-f509-4ec7-a04c-9999c82aaf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:24.917952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdf7e1-f509-4ec7-a04c-9999c82aaf9d admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:24.917952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdf7e1-f509-4ec7-a04c-9999c82aaf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:24.917952 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdf7e1-f509-4ec7-a04c-9999c82aaf9d admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:24.918413 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cdf7e1-f509-4ec7-a04c-9999c82aaf9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:24.919618 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cdf7e1-f509-4ec7-a04c-9999c82aaf9d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:24.931135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1446/2895] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:19:24 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:24.935609 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d94a8-1371-48c1-9e54-cf5fccb8a0ff admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:24.945487 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d94a8-1371-48c1-9e54-cf5fccb8a0ff admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:24.950803 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b711a53-5c36-4efe-8d97-a146a5e04d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:24.953617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b711a53-5c36-4efe-8d97-a146a5e04d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:24.953617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b711a53-5c36-4efe-8d97-a146a5e04d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:24.953617 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b711a53-5c36-4efe-8d97-a146a5e04d46 admin admin] PATH_INFO: `/v3/projects/473436ba36774a47b26c29e39e989c54/users/1b908ec5e79047769b817f2a2e35b85c/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:24.977451 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b711a53-5c36-4efe-8d97-a146a5e04d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=473436ba36774a47b26c29e39e989c54, user_id=1b908ec5e79047769b817f2a2e35b85c, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:24.980165 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b711a53-5c36-4efe-8d97-a146a5e04d46 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:25.036728 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1450/2896] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:19:24 2022] PUT /identity/v3/projects/473436ba36774a47b26c29e39e989c54/users/1b908ec5e79047769b817f2a2e35b85c/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:25.041314 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdf7e1-f509-4ec7-a04c-9999c82aaf9d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:25.051955 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdf7e1-f509-4ec7-a04c-9999c82aaf9d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:25.114760 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3e71e-cf69-434e-916c-0c0d73ba00bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:25.115520 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3e71e-cf69-434e-916c-0c0d73ba00bf admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:25.115800 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3e71e-cf69-434e-916c-0c0d73ba00bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:25.116072 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3e71e-cf69-434e-916c-0c0d73ba00bf admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:25.116802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3e71e-cf69-434e-916c-0c0d73ba00bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:25.118965 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3e71e-cf69-434e-916c-0c0d73ba00bf admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:25.129623 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1447/2897] 10.222.0.121 () {58 vars in 1126 bytes} [Thu May 26 18:19:25 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:19:25.134811 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b711a53-5c36-4efe-8d97-a146a5e04d46 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:25.145542 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b711a53-5c36-4efe-8d97-a146a5e04d46 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:25.150879 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef51ca-0e7b-4ada-967d-71a8240ca7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:25.151792 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef51ca-0e7b-4ada-967d-71a8240ca7ba admin admin] REQUEST_METHOD: `PUT` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:25.152065 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef51ca-0e7b-4ada-967d-71a8240ca7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:25.152335 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef51ca-0e7b-4ada-967d-71a8240ca7ba admin admin] PATH_INFO: `/v3/projects/473436ba36774a47b26c29e39e989c54/users/1b908ec5e79047769b817f2a2e35b85c/roles/4f574961798246a4b12ad2276e67685d` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:25.154364 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef51ca-0e7b-4ada-967d-71a8240ca7ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=473436ba36774a47b26c29e39e989c54, user_id=1b908ec5e79047769b817f2a2e35b85c, role_id=4f574961798246a4b12ad2276e67685d)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:25.157266 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef51ca-0e7b-4ada-967d-71a8240ca7ba admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:25.169818 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1451/2898] 10.222.0.121 () {60 vars in 1487 bytes} [Thu May 26 18:19:25 2022] PUT /identity/v3/projects/473436ba36774a47b26c29e39e989c54/users/1b908ec5e79047769b817f2a2e35b85c/roles/4f574961798246a4b12ad2276e67685d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:19:30.777233 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb3687-9c0b-4f81-9eaa-04cc5e0abd2c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:30.777750 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb3687-9c0b-4f81-9eaa-04cc5e0abd2c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:30.777863 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb3687-9c0b-4f81-9eaa-04cc5e0abd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:30.807739 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-05eb3687-9c0b-4f81-9eaa-04cc5e0abd2c None None] MFA Rules not processed for user `1b908ec5e79047769b817f2a2e35b85c`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:30.808497 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-05eb3687-9c0b-4f81-9eaa-04cc5e0abd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:30.858389 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:30.889301 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1448/2899] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:30.903477 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef51ca-0e7b-4ada-967d-71a8240ca7ba admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:30.905262 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-93ef51ca-0e7b-4ada-967d-71a8240ca7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:30.996055 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:31.005625 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef51ca-0e7b-4ada-967d-71a8240ca7ba admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:31.010870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaeb2e8-cd6f-4d68-91a7-429af7204767 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '683c54f489f440b9842cb94980214df1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:31.011541 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaeb2e8-cd6f-4d68-91a7-429af7204767 service glance] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:31.014950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaeb2e8-cd6f-4d68-91a7-429af7204767 service glance] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:31.014950 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaeb2e8-cd6f-4d68-91a7-429af7204767 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:31.074080 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaeb2e8-cd6f-4d68-91a7-429af7204767 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:31.075787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaeb2e8-cd6f-4d68-91a7-429af7204767 service glance] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:31.085752 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1452/2900] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:19:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:31.326150 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97085cfc-0011-464c-b027-625c009f746e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:31.326465 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97085cfc-0011-464c-b027-625c009f746e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:31.326885 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97085cfc-0011-464c-b027-625c009f746e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:31.380749 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-97085cfc-0011-464c-b027-625c009f746e None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:31.381972 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-97085cfc-0011-464c-b027-625c009f746e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:31.454013 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1449/2901] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 18:19:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:31.617056 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaeb2e8-cd6f-4d68-91a7-429af7204767 service glance] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:31.619020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-bcaeb2e8-cd6f-4d68-91a7-429af7204767 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:31.720441 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaeb2e8-cd6f-4d68-91a7-429af7204767 service glance] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:31.725601 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4edee-4ed3-49fa-9ed2-4345659ecfb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:31.726252 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4edee-4ed3-49fa-9ed2-4345659ecfb3 service swift] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:31.726537 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4edee-4ed3-49fa-9ed2-4345659ecfb3 service swift] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:31.726812 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4edee-4ed3-49fa-9ed2-4345659ecfb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:31.736097 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4edee-4ed3-49fa-9ed2-4345659ecfb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:31.737682 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4edee-4ed3-49fa-9ed2-4345659ecfb3 service swift] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:31.746358 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1453/2902] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:19:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:19:41.451834 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=50 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:41.743281 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=70 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:42.049447 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acac34-a0af-41c4-98ac-b3b3b58c22a8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:42.049737 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acac34-a0af-41c4-98ac-b3b3b58c22a8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:42.050018 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acac34-a0af-41c4-98ac-b3b3b58c22a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:42.068836 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-11acac34-a0af-41c4-98ac-b3b3b58c22a8 None None] MFA Rules not processed for user `e88e293efd2442d586abc01e621ac415`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:42.070112 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-11acac34-a0af-41c4-98ac-b3b3b58c22a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:42.077899 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1450/2903] 10.222.0.121 () {58 vars in 962 bytes} [Thu May 26 18:19:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:42.770652 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4edee-4ed3-49fa-9ed2-4345659ecfb3 service swift] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:42.829891 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4edee-4ed3-49fa-9ed2-4345659ecfb3 service swift] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:42.833646 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94835b-8a83-49f8-a12a-c827db3d786c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:42.834297 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94835b-8a83-49f8-a12a-c827db3d786c service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:42.834575 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94835b-8a83-49f8-a12a-c827db3d786c service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:42.834870 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94835b-8a83-49f8-a12a-c827db3d786c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:42.837388 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-9f94835b-8a83-49f8-a12a-c827db3d786c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:42.911787 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94835b-8a83-49f8-a12a-c827db3d786c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:42.913355 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94835b-8a83-49f8-a12a-c827db3d786c service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:42.925270 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1454/2904] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:19:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:45.683932 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ef406-f8e2-4afb-a8e7-f58ff946fbbb None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:45.684486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ef406-f8e2-4afb-a8e7-f58ff946fbbb None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:45.684486 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ef406-f8e2-4afb-a8e7-f58ff946fbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:45.703202 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c42ef406-f8e2-4afb-a8e7-f58ff946fbbb None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:19:45.704477 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c42ef406-f8e2-4afb-a8e7-f58ff946fbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:45.712889 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1451/2905] 10.222.0.121 () {58 vars in 1014 bytes} [Thu May 26 18:19:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:19:45.726081 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94835b-8a83-49f8-a12a-c827db3d786c service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:45.738539 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94835b-8a83-49f8-a12a-c827db3d786c service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:45.743762 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069440a-7a5c-47d8-a2ee-5cbba63b424a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:45.744463 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069440a-7a5c-47d8-a2ee-5cbba63b424a service swift] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:45.744746 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069440a-7a5c-47d8-a2ee-5cbba63b424a service swift] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:45.745020 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069440a-7a5c-47d8-a2ee-5cbba63b424a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:45.753969 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069440a-7a5c-47d8-a2ee-5cbba63b424a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:45.755675 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069440a-7a5c-47d8-a2ee-5cbba63b424a service swift] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:45.764580 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1455/2906] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:19:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:19:46.230122 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ef406-f8e2-4afb-a8e7-f58ff946fbbb None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:19:46.244802 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ef406-f8e2-4afb-a8e7-f58ff946fbbb None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:19:46.248884 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421e66d-ed8e-4e9c-9522-ebd77d53e32a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:19:46.249804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421e66d-ed8e-4e9c-9522-ebd77d53e32a service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:19:46.250147 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421e66d-ed8e-4e9c-9522-ebd77d53e32a service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:19:46.250492 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421e66d-ed8e-4e9c-9522-ebd77d53e32a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:19:46.252993 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2421e66d-ed8e-4e9c-9522-ebd77d53e32a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:19:46.271393 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421e66d-ed8e-4e9c-9522-ebd77d53e32a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:19:46.275727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421e66d-ed8e-4e9c-9522-ebd77d53e32a service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:19:46.295088 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1452/2907] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:19:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:55.760987 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=60 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:19:56.288379 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=12 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:18.873896 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069440a-7a5c-47d8-a2ee-5cbba63b424a service swift] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:18.875978 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d069440a-7a5c-47d8-a2ee-5cbba63b424a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:18.887816 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069440a-7a5c-47d8-a2ee-5cbba63b424a service swift] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:18.892075 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43d219-e84b-49bd-9276-763f582be634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:18.892732 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43d219-e84b-49bd-9276-763f582be634 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:18.893009 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43d219-e84b-49bd-9276-763f582be634 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:18.893282 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43d219-e84b-49bd-9276-763f582be634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:18.902674 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43d219-e84b-49bd-9276-763f582be634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:18.904117 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43d219-e84b-49bd-9276-763f582be634 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:18.914087 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1456/2908] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:20:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:20:23.626060 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421e66d-ed8e-4e9c-9522-ebd77d53e32a service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:23.627541 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2421e66d-ed8e-4e9c-9522-ebd77d53e32a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:23.649622 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421e66d-ed8e-4e9c-9522-ebd77d53e32a service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:23.658762 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d8858-8a2b-4d9a-b227-61a9212bf73c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:23.659458 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d8858-8a2b-4d9a-b227-61a9212bf73c service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:23.659744 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d8858-8a2b-4d9a-b227-61a9212bf73c service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:23.660017 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d8858-8a2b-4d9a-b227-61a9212bf73c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:23.662114 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c42d8858-8a2b-4d9a-b227-61a9212bf73c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:23.686758 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d8858-8a2b-4d9a-b227-61a9212bf73c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:23.691684 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d8858-8a2b-4d9a-b227-61a9212bf73c service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:23.709235 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1453/2909] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:20:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:20:28.909888 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=6 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:33.701509 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:54.391736 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43d219-e84b-49bd-9276-763f582be634 service neutron] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:54.411654 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43d219-e84b-49bd-9276-763f582be634 service neutron] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:54.415324 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4411926-e7b1-4f4e-95bf-3a8713418405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08709b462300437598c8db331e5ad20f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:54.415966 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4411926-e7b1-4f4e-95bf-3a8713418405 service neutron] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:54.416253 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4411926-e7b1-4f4e-95bf-3a8713418405 service neutron] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:54.416526 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4411926-e7b1-4f4e-95bf-3a8713418405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:54.418300 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-a4411926-e7b1-4f4e-95bf-3a8713418405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:54.447637 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4411926-e7b1-4f4e-95bf-3a8713418405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:54.449070 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4411926-e7b1-4f4e-95bf-3a8713418405 service neutron] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:54.457740 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1457/2910] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:20:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:21:04.453489 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=19 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:22:45.634584 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d8858-8a2b-4d9a-b227-61a9212bf73c service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:45.700818 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d8858-8a2b-4d9a-b227-61a9212bf73c service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:45.706771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a092011-be07-451e-bbc4-a0fd6a76ea06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0889a69a018452d8de1924072739d17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:45.707499 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a092011-be07-451e-bbc4-a0fd6a76ea06 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:45.707808 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a092011-be07-451e-bbc4-a0fd6a76ea06 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:45.708085 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a092011-be07-451e-bbc4-a0fd6a76ea06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:45.709893 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6a092011-be07-451e-bbc4-a0fd6a76ea06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:45.735487 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a092011-be07-451e-bbc4-a0fd6a76ea06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:45.737098 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a092011-be07-451e-bbc4-a0fd6a76ea06 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:45.748223 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1454/2911] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:22:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:22:47.803078 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65588cc-c162-4879-8dee-daa262084d29 None None] REQUEST_METHOD: `POST` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:47.803576 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65588cc-c162-4879-8dee-daa262084d29 None None] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:47.803687 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65588cc-c162-4879-8dee-daa262084d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:47.821323 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.auth.core [None req-d65588cc-c162-4879-8dee-daa262084d29 None None] MFA Rules not processed for user `485c784a043541619ffffc657b21ab97`. Rule list: `[]` (Enabled: `True`). {{(pid=92082) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:22:47.822495 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-d65588cc-c162-4879-8dee-daa262084d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:47.829431 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1458/2912] 10.222.0.121 () {58 vars in 972 bytes} [Thu May 26 18:22:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:22:47.869439 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a092011-be07-451e-bbc4-a0fd6a76ea06 service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:47.881852 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a092011-be07-451e-bbc4-a0fd6a76ea06 service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:47.886804 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e03452-45eb-4744-a74d-9647ac38516a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9ba9c73956403ba870f6f6b0e687aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:47.887476 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e03452-45eb-4744-a74d-9647ac38516a service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:47.887759 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e03452-45eb-4744-a74d-9647ac38516a service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:47.888038 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e03452-45eb-4744-a74d-9647ac38516a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:47.896362 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e03452-45eb-4744-a74d-9647ac38516a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:47.897874 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e03452-45eb-4744-a74d-9647ac38516a service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:47.905923 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1455/2913] 10.222.0.121 () {60 vars in 1389 bytes} [Thu May 26 18:22:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:22:55.974693 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65588cc-c162-4879-8dee-daa262084d29 None None] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:56.056059 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65588cc-c162-4879-8dee-daa262084d29 None None] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:56.060831 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:56.061722 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:56.061990 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:56.062265 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] PATH_INFO: `/v3/users/e88e293efd2442d586abc01e621ac415` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:56.063496 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e88e293efd2442d586abc01e621ac415)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:56.065400 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:56.282627 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e88e293efd2442d586abc01e621ac415'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:56.289505 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e88e293efd2442d586abc01e621ac415'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:56.290168 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e88e293efd2442d586abc01e621ac415'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:56.303248 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e88e293efd2442d586abc01e621ac415'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:56.374155 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e88e293efd2442d586abc01e621ac415'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:56.381421 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1459/2914] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:22:55 2022] DELETE /identity/v3/users/e88e293efd2442d586abc01e621ac415 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:22:56.610925 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e03452-45eb-4744-a74d-9647ac38516a service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:56.622262 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e03452-45eb-4744-a74d-9647ac38516a service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:56.681408 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:56.682263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:56.682543 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:56.683097 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] PATH_INFO: `/v3/projects/5950d9e178c2442cb0c62214d1eee60e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:56.684304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5950d9e178c2442cb0c62214d1eee60e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:56.686222 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:56.802459 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5950d9e178c2442cb0c62214d1eee60e', 'request_id': 'req-772b95cc-6ed5-4349-9e31-d45a1e224256'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:56.888741 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5950d9e178c2442cb0c62214d1eee60e', 'request_id': 'req-772b95cc-6ed5-4349-9e31-d45a1e224256'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:56.896012 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5950d9e178c2442cb0c62214d1eee60e', 'request_id': 'req-772b95cc-6ed5-4349-9e31-d45a1e224256'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:56.903795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:22:56.919848 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] The token cache is being invalidate because project 5950d9e178c2442cb0c62214d1eee60e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:22:56.920134 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:56.920751 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:56.922317 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1456/2915] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:22:56 2022] DELETE /identity/v3/projects/5950d9e178c2442cb0c62214d1eee60e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:22:56.926980 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:56.928391 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:56.942312 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f92a8d-95e2-4197-860e-bbbc189d9088 admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:56.990697 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:22:56.997417 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:56.998288 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:56.998573 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:56.998871 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] PATH_INFO: `/v3/users/1b908ec5e79047769b817f2a2e35b85c` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:57.000029 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b908ec5e79047769b817f2a2e35b85c)` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:57.002621 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:57.183371 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b908ec5e79047769b817f2a2e35b85c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:57.190140 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b908ec5e79047769b817f2a2e35b85c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:57.190800 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b908ec5e79047769b817f2a2e35b85c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:57.202360 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b908ec5e79047769b817f2a2e35b85c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:57.262301 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.notifications [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b908ec5e79047769b817f2a2e35b85c'} {{(pid=92082) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:57.268635 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1460/2916] 10.222.0.121 () {58 vars in 1228 bytes} [Thu May 26 18:22:56 2022] DELETE /identity/v3/users/1b908ec5e79047769b817f2a2e35b85c => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:22:57.512780 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:22:57.644304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:22:57.644304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b95cc-6ed5-4349-9e31-d45a1e224256 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:22:57.644304 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9845c1dd09f34a0aa117c5e212e20304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01a22dec10b499785fdfbdf0a6fcb47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:22:57.645912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:22:57.645912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:22:57.645912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] PATH_INFO: `/v3/projects/473436ba36774a47b26c29e39e989c54` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:22:57.645912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=473436ba36774a47b26c29e39e989c54)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:22:57.645912 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:22:57.759263 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '473436ba36774a47b26c29e39e989c54', 'request_id': 'req-709594b1-f62c-4dbe-9339-420303dd3051'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:57.836135 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '473436ba36774a47b26c29e39e989c54', 'request_id': 'req-709594b1-f62c-4dbe-9339-420303dd3051'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:57.842837 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '473436ba36774a47b26c29e39e989c54', 'request_id': 'req-709594b1-f62c-4dbe-9339-420303dd3051'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:57.854381 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] The token cache is being invalidate because project 473436ba36774a47b26c29e39e989c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:22:57.857110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:57.857110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:22:57.857110 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1457/2917] 10.222.0.121 () {58 vars in 1237 bytes} [Thu May 26 18:22:57 2022] DELETE /identity/v3/projects/473436ba36774a47b26c29e39e989c54 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:23:01.581272 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] Authenticating user token {{(pid=92082) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:23:01.582772 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:01.673626 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05ed1f-29b0-47fc-a28a-a8911b83d3ee admin admin] Validating token access rules against request {{(pid=92082) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:23:01.723379 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:01.733006 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514a6d4-f7e4-4988-bb50-4a3a220b9bfa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92082) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:23:01.737153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514a6d4-f7e4-4988-bb50-4a3a220b9bfa service placement] REQUEST_METHOD: `GET` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:23:01.737153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514a6d4-f7e4-4988-bb50-4a3a220b9bfa service placement] SCRIPT_NAME: `/identity` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:23:01.737153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514a6d4-f7e4-4988-bb50-4a3a220b9bfa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92082) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:23:01.737153 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.fernet_utils [None req-f514a6d4-f7e4-4988-bb50-4a3a220b9bfa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92082) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:23:01.748209 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514a6d4-f7e4-4988-bb50-4a3a220b9bfa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:23:01.749832 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514a6d4-f7e4-4988-bb50-4a3a220b9bfa service placement] RBAC: Authorization granted {{(pid=92082) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:23:01.760603 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: [pid: 92082|app: 0|req: 1461/2918] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:23:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:23:07.851249 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:23:11.816755 dv-cinder-836073-5-iscsi devstack@keystone.service[92082]: DEBUG dbcounter [-] [92082] Writing DB stats keystone:SELECT=6 {{(pid=92082) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:28:06.635398 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:28:06.637727 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:28:06.652795 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709594b1-f62c-4dbe-9339-420303dd3051 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:28:06.657833 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a6435-bd05-473f-a778-cdeed6554379 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7847d2c119474d49bef6029d51dbefdb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd61dea65b34b47a186151c6a63bc7d79', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:28:06.658478 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a6435-bd05-473f-a778-cdeed6554379 service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:28:06.658757 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a6435-bd05-473f-a778-cdeed6554379 service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:28:06.659058 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a6435-bd05-473f-a778-cdeed6554379 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:28:06.668335 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a6435-bd05-473f-a778-cdeed6554379 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:28:06.669987 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a6435-bd05-473f-a778-cdeed6554379 service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:28:06.680771 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1458/2919] 10.222.0.121 () {60 vars in 1370 bytes} [Thu May 26 18:28:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:28:16.675635 dv-cinder-836073-5-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}